Enable ULS logging during App deployment

Troubleshooting is always afterwards, meaning you need logging information to backtrack the origin of the errors. But normally the logging level is set to default and the actual interesting data is missing from the ULS logs.

This post provides an approach to enable logging before deploying your apps and reverting to the default afterwards.

Before making changes to the ULS diagnostic settings you want to backup/export your settings. Doing this from the Central Administration is not your preferred way or you must love to copy and past a lot.

LoggingInCentralAdmin

Export the diagnostic logging levels

You can easy export your settings by using PowerShell. Below a script which exports the diagnostic settings for the ULS logging levels. This script will export the logging levels to a CSV-file.

Get-SPLogLevel | Export-Csv "$($env:USERPROFILE)\Downloads\$((Get-Date).ToString("yyMMddHHmmss"))_LogLevels.csv" -NoTypeInformation

After opening the export in Excel, you get the data shown below.

ExportLoggingLevelsInExcel

Use PowerShell to get and/or update logging level

To get and/or update a particular log level via PowerShell you need to know how to scope the identity of the logging level. Once you know it, it’s pretty simple! The identifier is a combination of Area and Name.

Get-SPLogLevel -Identity "Search:General"

Change logging levels for App deployment

The general steps you want to include in your deployment scripts is based on lowering the logging for a predefined set of logging levels. Next execute the deployment and afterwards revert to the default.

  • Lower logging levels to Verbose
  • Execute deployment
  • Revert to default logging levels

Download the full PowerShell script here.

Lower the logging levels to Verbose

Below you find a script which sets a predefined set of logging levels. This set can be used as a basic for every deployment or configuration change. It is not set in stone that this is all you need, please update it to fit your needs.

Set-SPLogLevel -identity 'Search:Logging Correlation Data' -EventSeverity Verbose -TraceSeverity Verbose Set-SPLogLevel -identity 'SharePoint Foundation:Logging Correlation Data' -EventSeverity Verbose -TraceSeverity Verbose Set-SPLogLevel -identity 'SharePoint Server:Logging Correlation Data' -EventSeverity Verbose -TraceSeverity Verbose Set-SPLogLevel -identity 'SharePoint Foundation:Configuration' -EventSeverity Verbose -TraceSeverity Verbose Set-SPLogLevel -identity 'SharePoint Foundation:Database' -EventSeverity Verbose -TraceSeverity Verbose Set-SPLogLevel -identity 'SharePoint Foundation:Feature Infrastructure' -EventSeverity Verbose -TraceSeverity Verbose Set-SPLogLevel -identity 'SharePoint Foundation:General' -EventSeverity Verbose -TraceSeverity Verbose Set-SPLogLevel -identity 'SharePoint Foundation:Timer' -EventSeverity Verbose -TraceSeverity Verbose Set-SPLogLevel -identity 'SharePoint Foundation:Topology' -EventSeverity Verbose -TraceSeverity Verbose Set-SPLogLevel -identity 'SharePoint Foundation:Upgrade' -EventSeverity Verbose -TraceSeverity Verbose Set-SPLogLevel -identity 'SharePoint Foundation:PowerShell' -EventSeverity Verbose -TraceSeverity Verbose Set-SPLogLevel -identity 'SharePoint Foundation:App Deployment' -EventSeverity Verbose -TraceSeverity Verbose Set-SPLogLevel -identity 'SharePoint Foundation:App Management' -EventSeverity Verbose -TraceSeverity Verbose Set-SPLogLevel -identity 'SharePoint Foundation:App Marketplace' -EventSeverity Verbose -TraceSeverity Verbose Set-SPLogLevel -identity 'SharePoint Foundation:Application Authentication' -EventSeverity Verbose -TraceSeverity Verbose Set-SPLogLevel -identity 'SharePoint Foundation:Authentication Authorization' -EventSeverity Verbose -TraceSeverity Verbose Set-SPLogLevel -identity 'SharePoint Foundation:CSOM' -EventSeverity Verbose -TraceSeverity Verbose Set-SPLogLevel -identity 'SharePoint Foundation:CSOM Api' -EventSeverity Verbose -TraceSeverity Verbose Set-SPLogLevel -identity 'SharePoint Foundation:App Hosting Quota Management' -EventSeverity Verbose -TraceSeverity Verbose Set-SPLogLevel -identity 'SharePoint Foundation:App Monitoring' -EventSeverity Verbose -TraceSeverity Verbose Set-SPLogLevel -identity 'SharePoint Foundation:App Auth' -EventSeverity Verbose -TraceSeverity Verbose Set-SPLogLevel -identity 'SharePoint Portal Server:Runtime' -EventSeverity Verbose -TraceSeverity Verbose Set-SPLogLevel -identity 'SharePoint Portal Server:Long Running Operation' -EventSeverity Verbose -TraceSeverity Verbose Set-SPLogLevel -identity 'SharePoint Server:General' -EventSeverity Verbose -TraceSeverity Verbose Set-SPLogLevel -identity 'SharePoint Server:Setup and Upgrade' -EventSeverity Verbose -TraceSeverity Verbose Set-SPLogLevel -identity 'SharePoint Server:Database' -EventSeverity Verbose -TraceSeverity Verbose

Revert back to the default logging levels

Revert the logging levels to the default, after executing the deployment or configuration changes.

Clear-SPLogLevel

PowerShell x64 vs x86: set execution policy when using Visual Studio post build events

Running PowerShell scripts in the Post build events within a Visual Studio project? Getting errors telling you “running scripts is disabled on this system”?

Visual Studio Output window showing PowerShell execution error

While working on Office 365 Developer Patterns and Practices (PnP) solution OfficeDevPnP.PowerShell Commands I had troubles with building my solution. Visual Studio uses PowerShell x86 in the background instead of my default PowerShell which uses the x64 version.

Check PowerShell version: x64 versus x86

By running the following you can check the current version. Download the PowerShell function from Check-PSVersion.ps1

function Check-PSVersion { if ([System.IntPtr]::Size -eq 4) { # When equal to 4, then x86 Write-Host "Running x86 PowerShell session" } else { # When equal to 8, then x64 Write-Host "Running x64 PowerShell session" } } Check-PSVersion

Where to find the PowerShell executable for both x64 and x86

The x64 version of PowerShell (Windows 8.1) can be found under:

"C:\Windows\System32\WindowsPowerShell\v1.0\powershell.exe"

The x86 version of PowerShell (Windows 8.1) can be found under:

"C:\Windows\syswow64\WindowsPowerShell\v1.0\powershell.exe"

Change execution policy to use PowerShell in Visual Studio

Now we know we have two versions of PowerShell we can change the execution policy for the x86 version of PowerShell. Start the "C:\Windows\syswow64\WindowsPowerShell\v1.0\powershell.exe" with “Run as administrator”. Now change the execution policy accordantly.

Set-ExecutionPolicy Unrestricted

Show storage overview for site-collections with PowerShell

PowerShell is very powerful and helps you to quickly get different views on your SharePoint environment. The Usage class can be used to retrieve Storage information for a particular site-collection. Combining this into a PowerShell script provides an overview of storage sizing for your personal storage sites (My Sites) for example.

In this example all My Sites are located within the namespace http://my.contoso.com. When querying the My Sites I’m using this as input. After the basic query I transform the output to get the report.

The first query can be used for the numbers based on megabytes (MB). I’m using the System.Math.Round() method to transform the output based on custom tables in PowerShell.

Get-SPSite -Limit ALL | ?{$_.Url -like "*my.contoso.com*"} | Format-Table Url, @{Expression={[math]::round($_.Usage.Storage/1MB, 3)};Label="Storage"} -AutoSize

The output is shown below.

PowerShell console, output storage query

When I change the formatting for better readability you get the following query. The column Storage is displayed based on string formatting.

Get-SPSite -Limit ALL | ?{$_.Url -like "*my.contoso.com*"} | Format-Table Url, @{Expression={($_.Usage.Storage/1MB).ToString("#,###.000 MB")};Label="Storage"} -AutoSize

This will impact the column Storage shown in the output below.

PowerShell console, output storage query with string Formatting

Enjoy! Have fun with PowerShell.

How to: Detect the installed SKU of SharePoint with PowerShell

Do you need to know which SharePoint SKU or edition is installed on your server? Are you sure your product is not running the trial version? Do you need SharePoint 2010/2013 SKU and/or patch level information from your customer or IT department? This article provides you an easy to use PowerShell script which checks the installed products on a server and outputs the version (patch level).

You can use the Get-SPEdition PowerShell script as is. Simply run the script or use the Functions and call Get-SPEdition. Using the PowerShell scripts results in the following output.

Windows PowerShell: out Get-SPEdition

The script reads the registry to retrieve the installed versions. This information is located in below the “HKLM:software\Microsoft\Shared Tools\Web Server Extensions\15.0\WSS\InstalledProducts” path. Every GUID represents a SKU.

$products = @{ "BEED1F75-C398-4447-AEF1-E66E1F0DF91E" = "SharePoint Foundation 2010"; "1328E89E-7EC8-4F7E-809E-7E945796E511" = "Search Server Express 2010"; "B2C0B444-3914-4ACB-A0B8-7CF50A8F7AA0" = "SharePoint Server 2010 Standard Trial"; "3FDFBCC8-B3E4-4482-91FA-122C6432805C" = "SharePoint Server 2010 Standard"; "88BED06D-8C6B-4E62-AB01-546D6005FE97" = "SharePoint Server 2010 Enterprise Trial"; "D5595F62-449B-4061-B0B2-0CBAD410BB51" = "SharePoint Server 2010 Enterprise"; "BC4C1C97-9013-4033-A0DD-9DC9E6D6C887" = "Search Server 2010 Trial"; "08460AA2-A176-442C-BDCA-26928704D80B" = "Search Server 2010"; "84902853-59F6-4B20-BC7C-DE4F419FEFAD" = "Project Server 2010 Trial"; "ED21638F-97FF-4A65-AD9B-6889B93065E2" = "Project Server 2010"; "926E4E17-087B-47D1-8BD7-91A394BC6196" = "Office Web Apps 2010"; "35466B1A-B17B-4DFB-A703-F74E2A1F5F5E" = "Project Server 2013"; "BC7BAF08-4D97-462C-8411-341052402E71" = "Project Server 2013 Preview"; "C5D855EE-F32B-4A1C-97A8-F0A28CE02F9C" = "SharePoint Server 2013"; "CBF97833-C73A-4BAF-9ED3-D47B3CFF51BE" = "SharePoint Server 2013 Preview"; "B7D84C2B-0754-49E4-B7BE-7EE321DCE0A9" = "SharePoint Server 2013 Enterprise"; "298A586A-E3C1-42F0-AFE0-4BCFDC2E7CD0" = "SharePoint Server 2013 Enterprise Preview"; "D6B57A0D-AE69-4A3E-B031-1F993EE52EDC" = "Microsoft Office Online"; "9FF54EBC-8C12-47D7-854F-3865D4BE8118" = "SharePoint Foundation 2013" } $registryPath = "HKLM:software\Microsoft\Shared Tools\Web Server Extensions\$((Get-SPFarm).BuildVersion.Major).0\WSS\InstalledProducts" Get-RegistryKeyPropertiesAndValues -path $registryPath | ForEach-Object { Write-Host "Installed product: $($products.Get_Item($_.value)) (SKU ID: $($_.value))" } Write-Host "Installed version: $((Get-SPFarm).BuildVersion)"

With a little help from the Scripting Guy (Ed Wilson) and two MSDN articles I build the script. Enjoy!

Download: Get-SPEdition.ps1

Resources:

Ping, nslookup and ipconfig cmd via PowerShell

I’m used to regularly work with ping.exe, nslookup.exe and ipconfig.exe in the command prompt. With PowerShell becoming more and more the default it’s time to change my default behavior, sounds like a real programmer…

PowerShell 4.0 provides cmdlets for these commonly used command-line executables. Since I’m often searching for the cmdlet name, I thought to write a personal reminder and share it with you!

Another nice cmdlet to retrieve your IP addresses is:

SharePoint Client Browser 1.0 released, bye bye preview!

Finally after 2 months I decided to build the 1.0 version of SharePoint Client Browser and released it to the community! Although the preview (beta) status did not prevent people from downloading it. The counter is currently set at 555 downloads since start of the project on the 2nd of July (only 2 months ago).

CodePlex project and download at https://spcb.codeplex.com/.

So what got changed? I guess almost everything changed from authentication support for default (username and password), SharePoint Online, anonymous and forms based all the way to almost complete coverage of the Client Side Object Model (CSOM). That’s a bit over the top, but the basics for Foundation are in the tool. New capabilities for future releases will focus on Server components like taxonomy.

Remote PowerShell for SharePoint Online and on-premise

A hidden gem is the PowerShell support. It’s very easy to start a PowerShell session and use CSOM within PowerShell. Meaning remote PowerShell for SharePoint Online (and on-premise of course).

How to?
  • Open SharePoint Client Browser
  • Add a new site collection
  • Select the site collection in the tree view
  • Click the PowerShell-button in the menu bar (or use the context menu)
  • Enter your password (if needed)
  • Start using CSOM in PowerShell

PowerShell support for CSOM PowerShell console with CSOM and SharePoint Online (Office 365)

Some highlights of the tool

Why do you want to use this tool? One reason should be enough, it allows you to speed up your development by providing insights into the CSOM. And much more…

  • Get insight in your site collection structure
  • Find hidden lists, items or documents
  • Discover artifact properties
  • Easily start PowerShell, via context menu, and run (scripted) queries against your remote site collection
  • Support for both SharePoint 2010 and SharePoint 2013
  • Connect to on-premise or SharePoint Online (Office 365) site collections
  • No installer
  • Remote access from your desktop to site collection via Client Side Object Model (CSOM)
    • Can run remote, no need to run on the SharePoint server itself

SharePoint Client Browser for SharePoint 2013

Get disk storage for SharePoint databases

To get an indication for all database sizes in your SharePoint farm, run the following command. It shows the disk storage in bytes and MB.

image

The PowerShell script used for the output above, can be found below.

Add-PSSnapin Microsoft.SharePoint.PowerShell Get-SPDatabase | sort Name | ft Name, ` @{Name="DiskSizeRequired";Align='Right';Expression={[string]("{0:N0}" -f($_.DiskSizeRequired))}}, ` @{Name="Size(MB)";Align='Right';Expression={[string]("{0:N0}" -f($_.DiskSizeRequired/1mb))}}, ` Server -Autosize Get-SPDatabase | sort Name | select Name, DiskSizeRequired ` | measure-object -Property DiskSizeRequired -Sum -Maximum -Minimum -Average ` | ft Count, @{Name="Sum(MB)";Align='Right';Expression={[string]("{0:N0}" -f($_.Sum/1mb))}}, ` @{Name="Maximum(MB)";Align='Right';Expression={[string]("{0:N0}" -f($_.Maximum/1mb))}}, ` @{Name="Minimum(MB)";Align='Right';Expression={[string]("{0:N0}" -f($_.Minimum/1mb))}}, ` @{Name="Average(MB)";Align='Right';Expression={[string]("{0:N0}" -f($_.Average/1mb))}}

Using CSOM and PowerShell to query SharePoint Online or on-premise

Recently I released the SharePoint Client Browser (preview) that provides inside in a remote SharePoint site by using the Client Side Object Model (CSOM). The tool only reads data and shows it in a rich interface. When you want to make changes you have options: 1) build a custom tool and use the CSOM to interact with SharePoint or 2) use PowerShell and CSOM to interact with SharePoint.

This post shows how to use PowerShell and CSOM together to interact and automate configuration changes to your SharePoint. The example script can be used for both SharePoint Online and on-premise deployments.

This PowerShell script will be a new feature in the SharePoint Client Browser. This allows the user to start PowerShell directly from the SharePoint Client Browser and eliminates the need to do the scripting discussed in this post by yourself!

Prerequisites

Besides having a SharePoint site collection somewhere, you need to create a folder on your local machine. This folder needs to have the SharePoint Client Runtime assemblies. These can be downloaded here. Next to the SharePoint Client Runtime you need to place your scripts here (for example: OpenSite.ps1).

Folder (on local machine) with prerequisites

The PowerShell script

The easiest way to get started is creating a new TXT-file in the local folder and rename it to “OpenSite.ps1” (or choose a more appropriate name). Next right click with your mouse on the file and choose Edit. This will open up the Windows PowerShell ISE and you can start editing your script.

Authentication

The script is pretty easy and almost reusable for both SharePoint Online and on-premise, but it has a difference between them. Guest what it is?! The authentication is done differently. Below 3 different authentication examples.

Current user credentials

By not providing any credentials automatically the current user credentials are used.

SharePoint Online

When connecting to SharePoint Online you can use the new SharePointOnlineCredentials class. This makes it a lot easier then with SharePoint 2010.

$ctx = New-Object Microsoft.SharePoint.Client.ClientContext($siteUrl) $ctx.Credentials = New-Object Microsoft.SharePoint.Client.SharePointOnlineCredentials($loginname, $pwd)

SharePoint (on-premise) and credentials

Setting the credentials is done via the NetworkCredential class. That looks like this:

$ctx = New-Object Microsoft.SharePoint.Client.ClientContext($siteUrl) $ctx.Credentials = New-Object System.Net.NetworkCredential($loginname, $pwd)

The full example script

Below you can find the PowerShell script to use with SharePoint Online. Make sure when using the script you choose the right authentication model (see above). After running this script you can use the CSOM the same way as you would use CSOM in your C# application.

$loc = "C:\Users\Bram\Downloads\SharePoint" # Location of DLL's $siteUrl = "https://contoso.sharepoint.com" $loginname = "bram@contoso.onmicrosoft.com" Set-Location $loc Add-Type -Path (Resolve-Path "Microsoft.SharePoint.Client.dll") Add-Type -Path (Resolve-Path "Microsoft.SharePoint.Client.Runtime.dll") Write-Host "Please enter password for $($siteUrl):" $pwd = Read-Host -AsSecureString $ctx = New-Object Microsoft.SharePoint.Client.ClientContext($siteUrl) # Remove or replace line below to change authentication method $ctx.Credentials = New-Object System.Net.NetworkCredential($loginname, $pwd) $web = $ctx.Web $ctx.Load($web) $ctx.ExecuteQuery() Write-Host " Current web title is '$($web.Title)', $($web.Url)"

Slide deck and demos SharePoint Saturday 2013 Holland (#SPSNL13)

Last Saturday I presented at the SharePoint Saturday Holland 2013. It was a good vibe and the audience made it an interactive session. Below you can find the slide deck and parts of PowerShell scripts and source code which I used during my demos.

SharePoint Saturday Holland 2013

Developing hybrid SharePoint apps that run on-premise and in the cloud

With the new SharePoint App model running outside the SharePoint worker process it introduces new authentication models. As a developer you don’t want to build multiple versions of the same app implementing each authentication model separately. This session explains the differences between securing SharePoint apps with OAuth in Office 365 and S2S High Trust in on-premise deployments. You will learn how to build a single app that will run on-premise, online and hybrid SharePoint environments.

Demo: Configure the on-premise SharePoint with Trusted Security Token Issuer

To support high-trust (S2S) apps you need to configure the Trusted Security Token Issuer in your on-premise SharePoint farm. This script is used during the demo to configure the SharePoint farm.

Add-PSSnapin Microsoft.SharePoint.PowerShell -ea 0 # Get the .cer file that you want to use with your app. $certificate = Get-PfxCertificate "C:\Apps\AppCertSP2013.cer" # Get the issuer ID of your app. All the letters in the issuer ID GUID must be lowercase. $issuerId = 'a58e2347-0ead-4ba0-b4b7-75120aa09e4e' # Get the current authentication realm for your SharePoint site $realm = Get-SPAuthenticationRealm -ServiceContext "http://vm-sp-01/sites/dev" # Get the issuer ID together with the realm value. $fullIssuerIdentifier = $issuerId + '@' + $realm # Create a trusted security token service. This fetches metadata from your app (for example, the certificate) and establishes trust with it, so that SharePoint 2013 can accept tokens that are issued by your app. New-SPTrustedSecurityTokenIssuer -Name $issuerId -Certificate $certificate -RegisteredIssuerName $fullIssuerIdentifier –IsTrustBroker

Demo: Build Hybrid app with a single codebase for on-premise and cloud

My last demo was building a Hybrid app that consists of a single codebase which runs both on-premise and in the cloud. The logic which determines if SharePoint is hosted in on-premise or in the cloud is below.

public partial class Default : System.Web.UI.Page { protected void Page_Load(object sender, EventArgs e) { Uri hostWeb = new Uri(Request.QueryString["SPHostUrl"]); string contextTokenString = TokenHelper.GetContextTokenFromRequest(Request); if (string.IsNullOrEmpty(contextTokenString)) { using (var clientContext = TokenHelper.GetS2SClientContextWithWindowsIdentity(hostWeb, Request.LogonUserIdentity)) { clientContext.Load(clientContext.Web, web => web.Title); clientContext.ExecuteQuery(); Response.Write(clientContext.Web.Title); } } else { using (var clientContext = TokenHelper.GetClientContextWithContextToken(hostWeb.OriginalString, contextTokenString, Request.Url.Authority)) { clientContext.Load(clientContext.Web, web => web.Title); clientContext.ExecuteQuery(); Response.Write(clientContext.Web.Title); } } } }

Error installing Workflow Manager: Add-WFHost raises exception

Workflow Manager 1.0 (a.k.a. Azure Workflow) is the new workflow engine to support workflow in SharePoint 2013. This allows for a more scalable workflow engine which can be hosted on a separate (workflow) farm. The other option is hosting the workflow engine on the same server where SharePoint 2013 is hosted.

To deploy my environment(s) I use PowerShell. This is the same with configuring Workflow Manager 1.0. When running the Workflow Manager Configuration it generates the PowerShell command for you (see sample script).

Add-WFHost exception when configuring Workflow Manager

Although I use PowerShell scripts, it’s very likely to have the same issue and errors when running the Workflow Manager Configuration. When running my, slightly adjusted, script I got this error:

Add-WFHost : Could not successfully create management Service Bus entity 'WF_Management/WFTOPIC' with multiple retries within a timespan of 00:02:05.7093984.. The exception of the last retry is: The token provider service was not avaliable when obtaining a token for 'https://vm-sp-01.contoso.com:9355/WorkflowDefaultNamespace/$STS/Windows/'.. At C:\SPInstall\McwModules\mcwspinstall.wfm1.0\Ensure-WorkflowManager.ps1:117 char:19 + $wfHost = Add-WFHost -WFFarmDBConnectionString "Data Source=$wfDBServer; ... + ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + CategoryInfo : OperationStopped: (:) [Add-WFHost], TimeoutException + FullyQualifiedErrorId : WFRuntimeSettingFailed,Microsoft.Workflow.Deployment.Commands.AddWFHost

Solution to “token service provider not available” issue

It seems the service bus is not available, which is a local server address. I tried disabling the loopback adaptor, but that didn’t work out. After digging around some more I realized my environment has a proxy server. The checkbox “Bypass proxy server for local addresses” was unchecked in the Internet Settings (Control Panel » Internet Options » Connections tab » LAN settings).

After ensuring the local addresses are bypassed, the PowerShell script worked as a charm!