Log Retrevial Script v 2

My previous post included a Powershell script to download files using a .NET library. (This script was based on information from Bart de Smet’s blog.) The problem with this script was that it was taking too long to download files. For whatever reason, it took upwards of six hours to download 13MB. Obviously, that kind of performance is unacceptable. Before giving up on the project and looking for another way to solve the problem, I dug a bit deeper. In that research, I discovered that PowerShell 2.0 can use BITS to download files (What is BITS?).

To use this script you need to have PowerShell 2.0 installed. When that is installed, on Windows 2003 or higher, there will be a new folder created (C:\Windows\System32\WindowsPowershell\v1.0\Modules\BitsTransfer). To use the cmdlets from this module you need to import it.

Below you will see the new script along with the logging I’ve added since I’m scheduling the script.

# Script Constants
$now = Get-Date -format "yyyyMMdd_hhmm"
$strLogDir = "C:\Logs\"
$strLogFile = $strLogDir + "ScriptName_" + $now + ".log"
$BaseURL = "http://www.example.com/"
$BaseFilePrefix = "fileprefix_"
$Yesterday = Get-Date -format yyyyMMdd ((Get-Date).adddays(-1)) 
$HourArray = 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23
$BaseFileSuffix = "-0000.log.gz"
$SavePath = "C:\Log_Downloads\"

function StartLog
	{
	 $LogDate = Get-Date
	 if (!(Test-Path $strLogDir))
    	{
        	New-Item $strLogDir -type Directory | out-null
        	"==================== Start Log: ($LogDate) ===================="| Out-File $strLogFile
    	}
	 else
    	{
        	"==================== Start Log: ($LogDate) ===================="| Out-File $strLogFile
    	}
	 #Log Script Settings
	 "Script Settings" | Out-File $strLogFile -append 
	 "Base URL : $BaseURL" | Out-File $strLogFile -append
	 "Save Locaton: $SavePath" | Out-File $strLogFile -append
	}

function StopLog
	{
	 # Close Log File
	 $LogDate = Get-Date
	 "==================== End Log: ($LogDate) ===================="| Out-File $strlogfile -append
	}

#Begin Main Script
StartLog

Import-Module BitsTransfer

ForEach ($Hour in $HourArray) 
	{
	 if ($Hour -lt 10)
	 	{
	 	 $File = $BaseFilePrefix + $Yesterday + "0" + $Hour + $BaseFileSuffix
	 	}
	 else
	 	{
		 $File = $BaseFilePrefix + $Yesterday + $Hour + $BaseFileSuffix
		}
	 $URL = $BaseURL + $File
	 $SaveFile = $SavePath + $File
	 Start-BitsTransfer $URL $SaveFile
	 if (Test-Path $SaveFile)
	 	{
	 	 "     Successfully downloaded $URL to $SaveFile" | Out-File $strLogFile -append
		}
	 else
		{
		 "     File $URL did not transfer" | Out-File $strLogFile -append
		}
	}	

StopLog

Log Retrieval Script

Today’s project was to write a script that would retrieve log files using the HTTP protocol.  As I’ve said in my introductory post, I’m working on teaching myself PowerShell so this seemed like a great task for a script.  There are 24 log files per day so this script will get all 24 log files from the previous day. Here is the code:

# Log Retrevial Script
# Script Constants
$BaseURL = "http://www.example.com/"
$BaseFilePrefix = "fileprefix_"
$Yesterday = Get-Date -format yyyyMMdd ((Get-Date).adddays(-1))
$HourArray = 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23
$BaseFileSuffix = "-0000.log.gz"
$SavePath = "C:\Log_Downloads\"

$Client = New-Object System.Net.WebClient
Foreach ($Hour in $HourArray)
	{
	  if ($Hour -lt 10)
	 	{
	 	 $File = $BaseFilePrefix + $Yesterday + "0" + $Hour + BaseFileSuffix
	 	}
	 else
	 	{
		 $File = $BaseFilePrefix + $Yesterday + $Hour + $BaseFileSuffix
		}
	 $URL = $BaseURL + $File
	 $SaveFile = $SavePath + $File
	 $Client.DownloadFile($URL, $SaveFile)
	 Type $SaveFile
	}

An Introduction to me and this blog

This blog will chronicle my attempts to learn Microsoft’s Windows PowerShell.  I have over ten years of experience administering Microsoft Windows Server environments.  Along the way I have picked up experience with other technologies as well (Linux, Cisco IOS, Cisco VoIP, VMWare – just to name a few).

My day job involves administering many Windows 2008 boxes.  In the past, I’ve done most of my administrative work with minimal scripting. While that isn’t a problem with smaller environments, it obviously doesn’t scale well.  Given that Microsoft seems to be pushing Windows PowerShell as the way to script for their products, that seemed to be the logical tool for me to learn.  (I’m not a Microsoft fanboy and I realize there are lots of tools for scripting, some of which may be superior to PowerShell; however, this blog will focus on that technology unless I can’t figure out how to accomplish a task in PowerShell.

I’m also a firm believer in sharing code and other resources, so this blog is an attempt to share what I’ve figured out with others.  Feel free to use anything you find here that is useful for your own work.  All I ask is that if you find an error or bug, please let me know so I can fix my code.