PowerShell :: Third Tier

Archive for PowerShell

Feb
15

Updating your Exchange migration strategies

by dave

A couple of years ago, most SMB Exchange email migrations involved .pst files, or they were swing migrations that involved moving mailbox databases and remounting them, or moving mailboxes between one server and another. All that is changed now. Why? More and more, your clients will be moving back and forth between cloud and local storage, between hosted mail (whether that be Google Apps or Intermedia Hosted Exchange) and on-premises mail. Sometime you might have a multi-site corporation that wants to relocate. A growing percentage of the migrations being conducted now involve having to move data over WAN links, and that’s a game-changer.

So you have a migration planned, and you want to change MX records and cut over, but it looks like it’s going to take 5 days to migrate those 59gb of email down from the hosted provider! What do you do?

First, you think differently. No, I’m not talking about provisioning mac.com accounts, I’m asking us to stop thinking about the mail as a big chunk of data to be moved in one unit. Don’t even think of a single mailbox in that way. Start thinking about what part of the user’s mailbox they couldn’t live without for a day or two. The last two weeks of mail? The last two months? Once you’ve identified that interval that we’ll call “staple data”, make your plan: first cut over the MX record and immediately migrate that staple data. The reduced footprint of that data will turn your multi-day mailbox move effort into a several-hour effort. As soon as the staple data is in place on the new server, then begin the import of the rest of the data. I usually call that data the “historical backfill.”

Now how do you accomplish this date-delimited dissection of the mailbox? Pretty easily. All the basic tools we use or have used have all had date-specific parameters we can use during exports and imports. Exmerge had them, the PowerShell Export\Import-Mailbox commandlets have them. What are the other tools we can use? There’s one main other one that people aren’t usually familiar with. It’s called the Exchange Transporter Suite. It was built primarily to assist people in migrating from non-Microsoft mail platforms. In addition to tools to assist with Lotus migrations, it also has a very nice IMAP and POP migration tool. The IMAP tool is especially helpful for bringing client mail off of Google Apps and onto an Exchange server without having to do it from the Outlook client. Wouldn’t it be nice to do it in one place, all the mailboxes at once? Yep. It is. I’ll introduce you more formally to that tool in a future post.

How Come ExMerge Won’t Run?

One thing I’ve realized is that many SMB consultants aren’t sure how to move mail around in a world in which ExMerge isn’t easily accessible. That’s right, you can’t run ExMerge on an SBS 2008 or Exchange 2007 server to export\import mail. Ah, you might say, I’ve heard that you have to use PowerShell instead. Partly true, but you still can’t run the PowerShell import\export scripts on the server. You have to set up a separate management workstation to run them from. In fact, you can actually run ExMerge from a separate management workstation. The same change in functionality that will not let you run the PowerShell cmdlets on the server also “broke” ExMerge on the server.

The need for a separate management workstation to handle mail import\export procedures gets complicated for those who are trying to host their SBS 2008 box at a colo. In those situations, we’ve temporarily installed VMWare on the server and installed an XP Pro VM to use as a management workstation. Trying to handle the migration via a workstation connected to the colo via VPN is NOT a good idea.

So what’s involved in setting up a management workstation to run ExMerge or use the Import\Export scripts? I would suggest that you focus on running the PowerShell scripts rather than using ExMerge, but I’ll give you the details for both:

Installing ExMerge on a 32-bit Management Workstation

For ExMerge, you are going to follow the normal rules for running Exmerge on a workstation.

1. If on XP, install the IIS Snap-In component from the Add Components Add/Remove applet.
2. If on XP, install the Windows Server AdminPack tools.
3. Install the Exchange 2003 (yes, 2003) management tools on the workstation from the Exchange 2003 media. Ignore the schema error.
4. Download and install ExMerge on the workstation.
5. Pick or create a user who is not a member of the Domain Admins group to use for the import process.
6. Create an “ExMerge” group to assign Exchange permissions to and add the target user to the group.
7. Use the Exchange Management Console to delegate “Exchange View Only Administrator” control to the ExMerge group.

8. Give the ExMerge group Send-As and Receive-As rights. The easiest way to do this is in Exchange Powershell, like this:

Get-MailboxDatabase -identity “SERVER\First Storage Group\Mailbox Database” | Add-ADPermission -user “DOMAIN\ExMergeGroup” -ExtendedRights Receive-As, Send-As

(You’d swap SERVER and DOMAIN\ExMergeGroup with the appropriate server name and domain and group names in your environment.)

You should now be able to log on to that workstation as a member of the ExMerge group and run ExMerge to pull data out of Exchange 2007 or migrate data into Exchange 2007. I will show you how to do date-delimited extraction later in this article.

If you are interested in knowing whether ExMerge will work in the same way with Exchange 2010, the answer is no, but you can read more about that in this helpful article by Alexander Zammit.

Getting Ready to use Import-Mailbox on a 32-bit Management Workstation

If you are going to use PowerShell to pull the mail into the new Exchange 2007 server (which I recommend), then you are going to need to get a workstation ready to run the import process on. This is mainly written with an XP system in mind.

1. Install these prerequisites: .Net Framework 2.0 and its update, MMC 3.0 (if on XP), and Powershell 1.0. Ah yes, and Outlook 2007 or 2010.
2. Download, extract and install the Exchange 2007 32-bit Management Tools. Make sure you download the version that matches the service pack level of your Exchange 2007 installation, otherwise you will have problems.
3. Pick or create an account that is NOT a member of the Domain Admins or any other privileged group in the domain. Add that user to the local Administrators group on the Exchange 2007 server.
4. In the Exchange 2007 Management Console on the server (click on Organization), use the Add Exchange Administrator action to grant that user account the Exchange Server Administrator role.
5. To import and export mail, you need to grant import user account Receive-As and Send-As permission on any mailbox databases on the Exchange 2007 server. To do that, run this command at an “escalated” Exchange PowerShell prompt:

Get-MailboxDatabase | Add-ADPermission –User Importer –ExtendedRights Receive-As,Send-As
(replace Importer with the name of the user that you created\picked.)

6. Give the server some time to refresh its credential cache. If you don’t have up to a couple of hours to wait, you can restart the Information Store service on the Exchange 2007 server, and after the service restarts, the permissions will be in effect.
7. Now log on to the workstation using the account you chose/created, and open the Exchange Management Shell.
8. To use the import-mailbox cmdlet, adapt these examples for your use or look at the official docs:

To import all the .pst files in a specific directory into appropriate mailboxes, just make sure that each .pst file is named to match the user’s mailbox alias (jsmith’s .pst would be called jsmith.pst), and then execute something like this:

Dir D:\PSTDump\*.pst | Import-Mailbox

This will look at each .pst, match it to it’s mailbox on the server and import the contents of the .pst.

To import a single .pst file that doesn’t match the mailbox name, you can do this:

import-mailbox jsmith@domain.com -pstfolderpath d:\PSTDump\johnnyboy.pst

You can also do date-delimited imports, but that’s primarily something you’d want to do as an export process in ExMerge. I’ll go over that now.

Date-Delimited Mail Exports using ExMerge

Using date-delimited exports is commonly used when mailboxes are very large and need to be exported in chunks in order to avoid the 2gb limit on ANSI .pst files. It is also useful when there are many mailboxes to be moved in a short time and it is not practical to move all the data at once.

Let’s create a scenario: It’s March 13th, 2010 and tonight we need to migrate 115 users from an Exchange 2003 server in one domain to an Exchange 2007 server in another domain. We’ve already migrated the user accounts to the new domain using LDIFDE, but now it’s time to migrate the mail. We changed the firewall’s configuration an hour ago so that new email is being delivered to the mailboxes on the new server. We’ve set up our export\import account and are ready to use ExMerge.

Because the total amount of mail on the server is equal to 79gb (yes, this is an urgent project!) and a dozen of the mailboxes are larger than 3gb, it would take quite a bit of time to export all the mail out this evening. What we will do is export the “staple data” first: all the mail from January 1st 2010 up till today. Because that is only 73 days of mail, the .pst file for each mailbox will will be relatively small, and the entire export process will probably take 35 minutes or so. Since ExMerge will automatically name the .pst file after the user alias, the exported files are ready to be imported into Exchange 2007 immediately.

Once the initial run of .pst files has been exported and then imported successfully, it’s time to start working on historical backfill. We will go back to ExMerge and begin running another date-delimited export covering all of 2009, choosing a separate directory as the destination. When that export finishes, we will use import-mailbox again (pointing it at the new directory) to bring that mail into the mailboxes on the new server. We can go backwards, year by year, until all the mail has been brought in. Since the most critical data has already been imported, the migration should seem smooth to the users even if data is continuing to be imported over the next 24 hours. It is the rare user that refers to 36-month old emails on a daily basis.

What if you have just set up Exchange 2010? Can the management workstation you set up for importing into Exchange 2007 still be used? Nope, while the strategies I’ve outlined here are still valid, you can only use a system set up with Exchange 2007 management tools to import data into an Exchange 2007 mailbox server. If you have SP1 loaded on Exchange 2010, you can run the new-mailboximportrequest and new-mailboxexportrequest commands to export or import to\from .pst files. See this article for more details: http://msexchangeteam.com/archive/2010/04/26/454733.aspx

Using strategies like these can reduce the stress of having to migrate a lot of mailbox data in a short time-frame, and make large-mailboxes less formidable seeming. I hope you find this helpful. Of course, if you’re in the middle of it and short on patience, go ahead and open a ticket, and I’ll help you out!

—–

So who wrote this blog and what do they do for a living anyway?
We’re Third Tier. We provide advanced Third Tier support for IT Professionals.

Get Support BlogFeed

Blog

Twitter

Facebook

0 Categories : Dave Shackelford, Exchange, Migration, SBS 2008, Tips

Jan
29

Check for Free Space on your Drives – LazyAdmin PowerShell version

by edwin

When tasked to automate a task, I rely on my scripting background to get the job done. Back then, it was all VBScript. Now that more and more customers are moving to Windows Server 2008 servers, I try to convince my customers to use PowerShell for all their scripting needs.

A few weeks back, a friend of mine asked me to write a script that will check the disks on all of their servers and send a report via email. They have over a hundred or so servers in their data center and it will be a bit cumbersome to log in to each server just to check for free space on their disks. While their network operations center have access to reports generated by their monitoring tools, she didn't. Which is why she opted to do it on her own. I tried to demonstrate why PowerShell was my scripting language of choice for this task by highlighting how this task can be done in a single line of code. Not that it can't be done otherwise but this is an effective way to tell people how easy it is to use Windows PowerShell. Below is the script to do the task (BTW, this can be written in a single line. The blog engine just made it as it is due to space constraints)

Get-WmiObject Win32_Volume -computername "localhost" |

Select-Object __SERVER, Name, @{Name="Size(GB)";Expression={"{0:N1}" -f($_.Capacity/1gb)}},@{Name="FreeSpace(GB)";Expression={"{0:N1}" -f($_.freespace/1gb)}},@{Name="FreeSpacePerCent";Expression={"{0:P0}" -f($_.freespace/$_.capacity)}} |
Where-Object -FilterScript {$_.FreeSpacePerCent -lt 15} |
Sort-Object -property "FreeSpacePerCent"
Format-Table

Now this may seem intimidating at first but let's disect the script to understand what it is doing. The Get-WmiObject cmdlet calls the Win32_Volume class to scan thru a list of disks (I'm using the -computer parameter to highlight an important concept later on.) This includes mountpoints, local drives and USB drives. If you're only concerned about local disks and mountpoints, you can exclude USB drives by filtering via the DriveType attribute of the Win32_Volume class.

Get-WmiObject Win32_Volume -computername "localhost"

Since we are interested in the size of the disk, the free space in GB and in per cent values, we will use the Capacity and FreeSpace attributes of the Win32_Volume class. We do need to perform some calculations to make sure that we get the values we are accustomed with - GB for capacity and per cent in free space value. That's what the calculations are for, noting that capacity and free space are expressed in bytes. The Select-Object cmdlet simply creates a new object by defining a new attribute from the original Win32_Volume class - server hostname, drive or mountpoint name, capacity and free space - named FreeSpacePerCent. This will definitely give you all the disks on a server with the defined attributes.

Select-Object __SERVER, Name, @{Name="Size(GB)";Expression={"{0:N1}" -f($_.Capacity/1gb)}},@{Name="FreeSpace(GB)";Expression={"{0:N1}" -f($_.freespace/1gb)}},@{Name="FreeSpacePerCent";Expression={"{0:P0}" -f($_.freespace/$_.capacity)}}

But we don't want all of the drives. We only want those that have FreeSpacePerCent value less than your allowable threshold. In this example, less than 15 %. So, we use the Where-Object to filter the results.

Where-Object -FilterScript {$_.FreeSpacePerCent -lt 15}

As an administrator, we need to make sure that we address issues that are more critical than others. This is where the Sort-Object cmdlet comes in. We sort the results in order of increasing FreeSpacePerCent so we can immediately address those disks with very little free space left.

Sort-Object -property "FreeSpacePerCent"

Finally, the Format-Table cmdlet is just for aesthetics. The reason is because I will be sending the results of this script as an email attachment.

Now, that wasn't so hard, was it? Understanding what each cmdlet is doing and how you can pipe the results to another cmdlet is the key to maximizing the use of PowerShell. But if you have a hundred or more servers, you wouldn't want to copy the script on all of your servers and run it from there, would you? There are a few ways to accomplish this. One of which is to ue PowerShell Remoting. This will be another topic for a blog post as there is more to it than just simply writing and executing a PowerShell script remotely. What I opted to do here is simpler since I'm assuming that my friend nor I don't have access to Active Directory to create Group Policies to enable PowerShell Remoting on all of the servers. Since the Get-WmiObject cmdlet has a -computer parameter, I can use that to execute queries against remote computers. What I can do is simply read thru a list - probably a list of computers in Active Directory or even as simple as a text file list. I opted for the text file list for the same reason that I didn't go for PowerShell Remoting. So, what I did was to convert the script above to a function that I will call while passing values to the -computer parameter of the Get-WmiObject

function getDiskFreeSpace
{
Get-WmiObject Win32_Volume -computername $args |
Select-Object __SERVER, Name, @{Name="Size(GB)";Expression={"{0:N1}" -f($_.Capacity/1gb)}},@{Name="FreeSpace(GB)";Expression={"{0:N1}" -f($_.freespace/1gb)}},@{Name="FreeSpacePerCent";Expression={"{0:P0}" -f($_.freespace/$_.capacity)}} |
Where-Object -FilterScript {$_.FreeSpacePerCent -lt 15} |
Sort-Object -property "FreeSpacePerCent" |
Format-Table
}

I will call this function as I read thru a list of servers in a text file passing the computer names as parameters. And since I want the results of the query in a single output file, I used the Out-File cmdlet to save the results in a text file

ForEach($s in Get-Content C:\serverlist.txt)
{
getDiskFreeSpace $s | Out-File C:\diskFreeSpaceResults.txt -append
}

Sending emails

I used a simple script in the past to send emails via Windows PowerShell. This was prior to PowerShell v2.0

$SmtpClient = new-object system.net.mail.smtpClient
$SmtpServer = "smtp.yourmailserver.local"
$SmtpClient.host = $SmtpServer

$From = "Friendly Reminder "
$To = "recepient@yourmailserver.net"
$Title = "Subject Matter"
$Body = "Body Text"

$SmtpClient.Send($from,$to,$title,$Body)

With PowerShell v2.0, the Send-MailMessage cmdlet was made available to send an email message. This made it easier to integrate sending email functionalities in PowerShell scripts. Adding one more line to the script above, I've included email sending functionality with attachment

Send-MailMessage -to "recepient@mail.com" -from "sender@mail.com" -subject "Servers Disk Free Space Report" -Attachment "C:\diskFreeSpaceResults.txt" -SmtpServer "yourSMTPserver.mail.com"

Now, you can start using this script with a text file that lists all of your servers. Make sure that the server on which you will be running this script has Windows PowerShell v2.0 installed and that you can communicate with those servers. I've seen servers on different VLANs that are isolated from each other but are in the same Active Directory domain. This causes the script to fail due to connectivity issues. Check with your network administrator to be sure.

Categories : Edwin Sarmiento

May
1

Your poor man’s SQL Server log shipping-PowerShell version

by edwin

In 2008, I was privileged to be a part of a project to write a chapter for a book that was for a good cause. SQL Server MVP Deep Dives became an avenue for a lot of the SQL Server MVPs to share their expertise for the benefit of not just the technical community but of the beneficiary.

I wrote a chapter for the book based on this blog post some three years ago and one of the recommendations I did was to convert the VBScript scripts to Windows PowerShell. So, here it is.

I've converted the VBScript that does the checking of the folders containing my latest LOG backups and restoring them in NORECOVERY mode to Windows PowerShell. What is fascinating is he fact that if the process I've outlined in the blog post is followed thru, adding another database to be configured for log shipping was as easy as creating regular LOG backups via TSQL scripts or database maintenance plans - no need to add an entry for every database that will be configured. How cool was that?

So, here's the Windows PowerShell version of the custom restore LOG backup command

$logBackupFolder="your LOG backup location here or a parameter variable"

##Variable for time duration - the amount of time you need to generate and restore LOG backups
$1HourOld = [DateTime]::Now.AddHours(-1)

##Retrieve folders and files in the specified directory
foreach ($databaseName in (Get-ChildItem $logBackupFolder Where { $_.PsIsContainer }) )
{
foreach ($logName in (Get-ChildItem $logBackupFolder\$databasename where {($_.CreationTime -ge $1HourOld) -and ($_.Extension -eq ".trn")} SELECT name))
{

$logBackupPath = [System.String]::Concat("'$logBackupFolder\",$databasename,"\",$logName.Name,"'")
$restoreSQL = "RESTORE LOG $databaseName FROM DISK=$logBackupPath WITH NORECOVERY" Out-File D:\$databaseName.sql

Invoke-Expression "osql.exe -SYourSQLServerInstanceHere -E -iD:\$databaseName.sql"

}
}

You might be wondering why I am using osql.exe instead of sqlcmd.exe. Well, it's still there even if you have SQL Server 2008. Which means this script will work even for SQL Server 2000 instances as long as you install Windows PowerShell on the machine running SQL Server.

And all of that for a mere 17-liner script. I could write it even less than that but I'm a big fan of script readability. Now, that's a good enough reason to dig deeper on what Windows PowerShell has to offer

Categories : Edwin Sarmiento

Feb
25

Windows PowerShell Community Review

by steve

Received this request earlier today, so getting the word out!

The Windows PowerShell Community Review process is looking for volunteers for our third documentation review cycle – especially beginners and intermediate PowerShell users and people with little or no programming background. Volunteers can contact June at juneb@microsoft.com or Marco (Marco.Shaw@gmail.com ).

Windows PowerShell Community Review

Have you ever read Help that wasn't really helpful? Here's your chance to fix it.

The Windows PowerShell documentation team and PowerShellCommunity.org jointly sponsor the Windows PowerShell Community Doc Review. As a member, you'll get to read and comment on the Help docs before they're published, and work with the writers, editors, and the product team to make sure every word is really helpful.

We're looking for users at all experience levels and with all different backgrounds, but we love to have beginners, people with no programming experience, people who know other scripting languages or shells, and people who are not native English speakers. If you're a system admin and you don't really know Windows PowerShell, this is a great way to learn it with help from insiders.

Ready to rock the help? Contact June Blender (juneb@microsoft.com) or Marco Shaw (marco.shaw@gmail.com).

Thanks,

June

Windows Powershell Spoken Here

Categories : Steve

Archive for PowerShell

Updating your Exchange migration strategies

Check for Free Space on your Drives – LazyAdmin PowerShell version

Your poor man’s SQL Server log shipping-PowerShell version

Windows PowerShell Community Review

Search

Support