Make Windows PowerShell Your Web Client
Summary: Interact with websites from Windows PowerShell with this how-to post that wraps .NET Framework classes.
Hey, Scripting Guy! I would like to be able to download files and things from the Internet via Windows PowerShell. Is there an easy way to do this? I have not been able to find a Windows PowerShell cmdlet, and I would like to share whatever I come up with other users.
— BR
Hello BR,
Microsoft Scripting Guy Ed Wilson here. James Brundage began talking about wrapping .NET Framework classes to work with the web in yesterday’s Hey, Scripting Guy! post. That post is a great introduction to wrapping .NET Framework classes to work with the Internet. Let’s let him complete the discussion.
James Brundage is the founder of Start-Automating (http://www.start-automating.com), a company dedicated to saving people time and money by helping them automate. Start-Automating offers Windows PowerShell training and custom Windows PowerShell and .NET Framework development. James formerly worked on the Windows PowerShell team at Microsoft and is the author of the PowerShellPack, a collection of Windows PowerShell scripts to enable building user interfaces, interact with RSS feeds, retrieve system information, and more. You can follow James on Twitter @JamesBru.
A Series on Splatting (and Stuff)
Part 5: Splatting .NET Framework conclusion
In previous posts, we have covered how to build commands that wrap other commands with splatting and building bridges between two commands with splatting. Yesterday we looked at improving a Windows PowerShell script that invoked a .NET command. Today, we will learn how to apply those skills to make some usable Windows PowerShell scripts to replace some hard-to-use .NET.
Great! Let’s take the New-WebClient function that was developed yesterday, because we can think of the next command we will write as bridging New-WebClient and a quick function to invoke the method.
Our begin block will also contain this New-WebClient function and a helper function called Invoke-Method that lets us invoke .NET methods in a Windows PowerShell syntax.
The $BeginBlock variable will contain the two helper commands, and nothing else. If these helper commands were going to be used by a larger group of commands, I’d put them into the same module as the command I’m working on. However, you can also embed a private command this way. The $BeginBlock and its two functions are seen here.
$BeginBlock = {
function New-WebClient
{
[CmdletBinding(DefaultParameterSetName=”Easy”)]
param(
[string]$BaseAddress,
[Parameter(ParameterSetName=’Hard’)]
[Net.ICredentials]$Credentials,
[Parameter(ParameterSetName=’Easy’)]
[Management.Automation.PSCredential]$Credential,
[Parameter(ParameterSetName=’Hard’)]
[Net.WebHeaderCollection]$Headers,
[Parameter(ParameterSetName=’Easy’)]
[Hashtable]$Header,
[Parameter(ParameterSetName=’Hard’)]
[Collections.Specialized.NameValueCollection]$QueryString,
[Parameter(ParameterSetName=’Easy’)]
[Hashtable]$Query,
[Parameter(ParameterSetName=’Hard’)]
[bool]$UseDefaultCredentials,
[Parameter(ParameterSetName=’Easy’)]
[Switch]$Anonymous
)
process {
if ($psCmdlet.ParameterSetName -eq “Hard”) {
New-Object Net.Webclient -Property $psBoundParameters
} else {
$newWebClientParameters = @{} + $psBoundParameters$newWebClientParameters.UseDefaultCredentials = -not $Anonymous
$null = $newWebClientParameters.Remove(“Anonymous”)if ($newWebClientParameters.Credential) {
$newWebClientParameters.Credentials = $newWebClientParameters.Credential.GetNetworkCredential()
$null = $newWebClientParameters.Remove(“Credential”)
}if ($newWebClientParameters.Header) {
$newWebClientParameters.Headers = New-Object Net.WebHeadercollection
foreach ($headerPair in $newWebClientParameters.Header.GetEnumerator()) {
$null = $newWebClientParameters.Headers.Add($headerPair.Key, $headerPair.Value)
}
$null = $newWebClientParameters.Remove(“Header”)
}if ($newWebClientParameters.Query) {
$newWebClientParameters.QueryString = New-Object Collections.Specialized.NameValueCollection
foreach ($QueryPair in $newWebClientParameters.Query.GetEnumerator()) {
$null = $newWebClientParameters.QueryString.Add($QueryPair.Key, $QueryPair.Value)
}
$null = $newWebClientParameters.Remove(“Query”)
}
New-WebClient @newWebclientParameters
}
}
}
function Invoke-Method
{
param($Object, [string]$Method, $Parameters)
process {
$realMethod =$object.”${method}”
if ($realMethod) {
$realMethod.Invoke($Parameters)
}
}
}
}
The parameter block is next. Remember how we said that most of what you do in .NET can be expressed as creating a class, setting lots of properties, and running lots of methods? The begin block took care of the first two. The parameter block and process block will handle the rest.
You can think of any Windows PowerShell command as one or more things that you can do. Each parameter set may be a small variation on the thing. Many commands actually contain many related operations in one, with each parameter set representing a different operation.
When wrapping something from .NET in Windows PowerShell, it is easy to go down the list and turn methods into parameter sets.
Here are the .NET methods we want to wrap:
-
# string DownloadString(System.Uri address)
-
# byte[] DownloadData(System.Uri address)
-
# void DownloadFile(System.Uri address, string file)
-
# byte[] UploadData(System.Uri address, string method, byte[] data)
-
# byte[] UploadFile(System.Uri address, string method, string file)
-
# byte[] UploadString(System.Uri address, string method, string string)
-
# byte[] UploadValues(System.Uri address, string method, NameValueCollection values)
Turning these into parameters is tedious, but easy. Here is a list of guidelines that describe how we’ll do it.
-
Each method becomes a parameter set.
-
Each parameter becomes a mandatory parameter for that parameter set.
-
Any parameter that is in multiple methods will be in multiple parameter sets (it is easy, just add another [Parameter()] attribute).
-
Make sure that you give common parameters a position (i.e. address and method).
-
If there is a case in which the only way the methods are different is by the type of data that they return, make an -AsTYPE parameter to let the user decide (i.e. DownloadString and DownloadData).
-
Select the most common operation as the default parameter set.
-
Add comments for user help.
-
Add the ValueFromPipelineByPropertyName attribute to each parameter (unless you have a good reason not to, it opens up some interesting possibilities if you do).
If you are writing one of these from scratch in the Windows PowerShell Integrated Scripting Environment (ISE), you might want to try using the Add-Parameter function from IsePack (part of PowerShellPack) because it simplifies some repetitive typing. The Add-Parameter function has several switches that let you add mandatory parameters, select a parameter set, create the parameter from the pipeline or from the pipeline by property name.
Here is the parameter block for Send-WebRequest, which combines the ”Easy” parameter set of New-WebClient and the parameters that match each newly added operation.
$paramBlock = {
<#
.Synopsis
Sends web requests to download or upload information from the web
.Description
Uses the System.Net.WebClient class to download or upload information from the web
.Example
Send-WebRequest “http://www.start-automating.com/”
#>
[CmdletBinding(DefaultParameterSetName=”DownloadString”)]
param(
# The Uri to download or upload
[Parameter(ParameterSetName=’DownloadData’,
Position=0,
Mandatory=$true,
ValueFromPipelineByPropertyName=$true)]
[Parameter(ParameterSetName=’DownloadString’,
Position=0,
Mandatory=$true,
ValueFromPipelineByPropertyName=$true)]
[Parameter(ParameterSetName=’DownloadFile’,
Position=0,
Mandatory=$true,
ValueFromPipelineByPropertyName=$true)]
[Parameter(ParameterSetName=’UploadData’,
Position=0,
Mandatory=$true,
ValueFromPipelineByPropertyName=$true)]
[Parameter(ParameterSetName=’UploadFile’,
Position=0,
Mandatory=$true,
ValueFromPipelineByPropertyName=$true)]
[Parameter(ParameterSetName=’UploadString’,
Position=0,
Mandatory=$true,
ValueFromPipelineByPropertyName=$true)]
[Parameter(ParameterSetName=’UploadValues’,
Position=0,
Mandatory=$true,
ValueFromPipelineByPropertyName=$true)]
[Uri]
$Address,
# The method used to send an upload request (i.e. Get or Post)
[Parameter(ParameterSetName=’UploadData’,Position=1,Mandatory=$true,ValueFromPipelineByPropertyName=$true)]
[Parameter(ParameterSetName=’UploadFile’,Position=1,Mandatory=$true,ValueFromPipelineByPropertyName=$true)]
[Parameter(ParameterSetName=’UploadString’,Position=1,Mandatory=$true,ValueFromPipelineByPropertyName=$true)]
[Parameter(ParameterSetName=’UploadValues’,Position=1,Mandatory=$true,ValueFromPipelineByPropertyName=$true)]
[string]$Method = “Get”,# The data to upload
[Parameter(ParameterSetName=’UploadData’,Mandatory=$true,ValueFromPipelineByPropertyName=$true)]
[Byte[]]$UploadData,
# The file to upload
[Parameter(ParameterSetName=’UploadFile’,Mandatory=$true,ValueFromPipelineByPropertyName=$true)]
[string]$UploadFile,
# The string to upload
[Parameter(ParameterSetName=’UploadString’,Mandatory=$true,ValueFromPipelineByPropertyName=$true)]
[string]$UploadString,
# The string to upload
[Parameter(ParameterSetName=’UploadValues’,Mandatory=$true,ValueFromPipelineByPropertyName=$true)]
[Hashtable]$UploadValue,
# If set, the information will be returned as Data
[Parameter(ParameterSetName=’DownloadData’,Mandatory=$true,ValueFromPipelineByPropertyName=$true)]
[Switch]$ASData,
# If set, the infromation will be downloaded to a file
[Parameter(ParameterSetName=’DownloadFile’,Mandatory=$true,ValueFromPipelineByPropertyName=$true)]
[String]
$ToFile,
# The Base Address. The Uri will be appended to this address
[Parameter(ValueFromPipelineByPropertyName=$true)]
[string]$BaseAddress,# The Credential used to send the web request
[Parameter(ValueFromPipelineByPropertyName=$true)]
[Management.Automation.PSCredential]$Credential,# A collection of header information
[Parameter(ValueFrom PipelineByPropertyName=$true)]
[Hashtable]$Header,# A set of data for GET or POST query
[Parameter(ValueFromPipelineByPropertyName=$true)]
[Hashtable]$Query,# If this is set, then the connection will be anonymous. Otherwise, the default credentials will be used.
[Parameter(ValueFromPipelineByPropertyName=$true)]
[Switch]$Anonymous
)
}
It’s a piece of cake right? (Well maybe the kind that Marie Antoinette would serve).
The process block is less tedious, but follows a similar pattern. The start of the command becomes a bit of common code (creating the web request) followed by either a switch statement (if you prefer readability) or lots of else-ifs (if you prefer speed) that set up the input for the method, followed by a calling the method.
Because the parameter set name is the method name, and the webclient is always the object, the only thing that has to be in the switch statement is the part that determines the arguments. Here’s what our process block looks like.
$processBlock = {
# Create the web client
$newWebClientParameters = @{}
foreach ($p in “BaseAddress”, “Credential”, “Header”, “Query”, “Anonymous”) {
if ($psBoundParameters[$p]) {
$newWebClientParameters[$p] = $psBoundParameters[$p]
}
}
$webClient = New-WebClient @newWebClientParameters
$parameters = @()
switch ($psCmdlet.ParameterSetName)
{
DownloadString {
$parameters = $Address
}
DownloadData {
$parameters = $Address
}
DownloadFile {
$unresolvedFile = $psCmdlet.SessionState.Path.GetUnresolvedProviderPathFromPSPath($ToFile)
$parameters = $Address, $unresolvedFile
}
UploadData {
$parameters = $Address, $Method, $UploadData
}
UploadFile {
$resolvedFile = $psCmdlet.SessionState.Path.GetResolvedProviderPathFromPSPath($uploadFile)
$parameters = $Address, $Method, $resolvedFile
}
UploadString {
$parameters = $Address, $Method, $UploadString
}
UploadValues {
$nameValueCollection = New-Object Collections.Specialized.NameValueCollection
foreach ($pair in $value.GetEnumerator()) {
$null = $nameValueCollection.Add($Pair.Key, $Pair.Value)
}
$parameters = $Address, $Method, $nameValueCollection
}
}
$invokeMethodParameters = @{
Object = $webClient
Method = $psCmdlet.ParameterSetName
Parameters = $parameters
}
Invoke-Method @invokeMethodParameters
}
Now let’s create our function from the three different parts we just finished making. The Send-WebRequest function is seen here.
$functionDefinition = “function Send-WebRequest {
$paramBlock
begin {
$beginBlock
}
process {
$processBlock
}
}
“$functionDefinition | Set-Content .\Send-WebRequest.ps1
. .\Send-WebRequest.ps1Let’s try downloading the Hey, Scripting Guy blog:
$rss = Send-WebRequest “http://blogs.technet.com/b/heyscriptingguy/rss.aspx”
$rss = [xml]$rss
$rss.rss.channel.item | Select-Object Title, PubDate
It even has help! Cool!
Get-Help Send-WebRequest –Full
The complete Send-WebRequest function can be downloaded from the Scripting Guys Script Repository. You can paste the function into the Windows PowerShell ISE, run the script, and then use the code seen here to download the RSS feed for the Hey, Scripting Guy! Blog.
$rss = Send-WebRequest “http://blogs.technet.com/b/heyscriptingguy/rss.aspx”
$rss = [xml]$rss
$rss.rss.channel.item | Select-Object Title, PubDate
When you run this code, the output shown in the following image appears.
If you save the Send-WebRequest function to a Windows PowerShell script file, you can dot-source it into your new script. This is seen in the UseSendWebRequestFunction.ps1 script.
UseSendWebRequestFunction.ps1
. C:\fso\Send-WebRequestFunction.ps1
$rss = Send-WebRequest “http://blogs.technet.com/b/heyscriptingguy/rss.aspx”
$rss = [xml]$rss
$rss.rss.channel.item | Select-Object Title, PubDate
BR, that is all there is to using Windows PowerShell to wrap .NET Framework commands. Guest blogger week has ended. Thank you James for all this information and scripts. Join us tomorrow for the Weekend Scripter.
We invite you to follow us on Twitter or Facebook. If you have any questions, send email to us at scripter@microsoft.com or post them on the Official Scripting Guys Forum.. See you tomorrow. Until then, peace.
Ed Wilson and Craig Liebendorfer, Scripting Guys
Light
Dark
0 comments