SharePoint Online

Crawling as fast as possible

More and more we are becoming a generation of searchers. And that is not meant in a philosophical, running your fingers through your beard kind of way, but in a very practical everyday reality kind of way. When was the last time an argument you had among friends was not settled by searching the internet for the right answer? And this trend is not just influencing your personal life, but your work life as well. A clear shift is happening within corporations to go to a flatter organization structure, have self-organizing teams and increased cross functional interaction. We are trading in hierarchies for communities. Heck, there are even companies that let their employees pick their own job titles[1].

In this world of less structure one thing becomes more and more important to still be able to do your work: search! However, making sure stuff is available to be found within SharePoint Online is not always straightforward.

Our colleagues Marijke Ursem and Martin Offringa wrote a blog (read it here) about the workings of search in SharePoint and how to make sure the search results are shown just how you like it. So we will not cover any of that here. Instead we will dive into the bag of tricks we have to make sure that content is searchable as quickly as possible.

The Index

For those of you who are new to the subject of search in SharePoint, let us quickly cover some of the basics.

The search results you see in the content search web part or the search results web part are not coming directly from your lists and libraries, but from the search index. The index can be considered as one big bucket with all the searchable content and only stuff that is in the index can found through search.

Based on an automated schedule the index is filled with the latest changes that occurred in your tenant. This is done by the crawl, and in SharePoint Online there are two variants of the crawl: 1) the continuous crawl that runs every 15 minutes and picks up new and changed documents or items and 2) the incremental crawl that runs every 4 hours and picks up changes in the search configuration.

Crawling 1

Schematic to show how the content a tenant, the crawl and the index relate to each other.

Lack of Control

One of the most heard complaints about search in SharePoint Online related to search is that even with the highest of permission levels on your tenant, you are still not fully in control of the crawl. This is because, in contrast with an On Premise situation, the automated schedule cannot be changed. In SharePoint Online, it is Microsoft who runs the show.

But there is no use in complaining, because at the moment there is no option to speed up the crawls. So if you can’t beat them, join them. Because there are some tricks that help you to go as fast as possible when it comes to having your changes crawled in SharePoint Online.

The Basics

First, a document which has no published version will not be crawled. So when you are working inside a document library that has minor (concepts) and major (publications) versioning activated, make sure to publish your documents.

Second, when you add a column to a list of library it will not be crawled if there is no item that has a value for that column. So make sure that at least one item contains a value for this new column, even if that means adding a temporary test item.

The Simple Tricks

Maybe the best analogy for the scheduled crawls is to view them as an old fashioned postman, who is doing his rounds on a fixed schedule. And on his round he comes by a series of classic postboxes with the little red flags on them. The classic postbox works by raising the flag when there is something in it and leaving it down when it is empty. And let’s decide that in this analogy raising the flag is a signal to the postman doing his rounds to empty the postbox.

Furthermore, it is important to know that the crawl acts on value changed. So in our analogy, value changes raise the flag automatically and indicate to our postman to pick up the changes. So if you have a document and you change for example the person mentioned in the “owner” field then this change will automatically be picked up by our postman. However, when you change the way the owner needs to be presented from “account name” to “display name with presence” this change will not automatically raise the flag since no value change occurred. Only a setting change was done.

To make sure your change is picked up anyhow, you can raise the flag yourself via the library settings, which is described in a support article of Microsoft[2].

Crawling 2

The same article also describes how to raise the flag for a whole site and since Microsoft already did an excellent job of explaining how it is done, we have nothing to add to their story.

Crawling 3

When we leave the libraries and list behind us and start getting our hands dirty within the search center of SharePoint Online there are also some tricks we can pull out of our top hat. Within the Search Schema we can have a ball setting up managed properties and mapping all sorts of crawled properties to our managed properties.

For those of you who are new to the Search Schema, crawled properties and managed properties and want to learn more about the topic, we recommend to give the support article Manage the search schema in SharePoint Online a good read.

While you can do a lot of nice and necessary work inside the Search Schema, you will have to do something extra to make sure your changes have effect. The reasoning behind this is that “…Because your changes are made in the search schema, and not to the actual site, the crawler will not automatically re-index the site”[3]. What you need to do is re-index the site which uses the crawled property that you have used in your managed property mapping and then “…site content will be re-crawled and re-indexed so that you can start using the managed properties in queries, query rules and display templates”. Or if the crawled property is only attached to a certain library or list, you can re-index that list which will have the effect that “…all of the content in that library or list is marked as changed, and the content is picked up during the next scheduled crawl and re-indexed”.

So for sites, lists and libraries we have the power to raise the flag and our postman (a.k.a. crawl) will pick up our changes and update them in the index so they are seen in search results.

 

The Advanced Tricks

At this point your question will undoubtedly be what else you can do to give the crawl a kick, because going into every site that you want to raise the flag for one by one is just too much of a hassle.

Well unfortunately, this raising the flag thing is the only instrument we have in the wen interface. Because as said, there is no way to influence the schedule, only ways to influence what is picked up during the next round of our postman. But rest assured, we are not suggesting that you actually go into every site that you have and click a button. We are suggestion that you put others to work for you.

The first option you have is to put Microsoft to work for you. Via the Admin Center you can raise a ticket to Microsoft technical support and ask them to re-index a bunch of sites, a site collection or even all your site collections. It is also possible to request a re-index of all the user profiles. What Microsoft Technical Support then will do is raise the flag for all your content so that everything gets picked up during the next round of the postman. Upside is that Microsoft can do this much more efficient, but the downside is that you still have to wait for the next incremental crawl. And of course, there is waiting involved between raising the ticket and getting a response from Microsoft.

So, where do we have to turn to get even faster results? This is really not a question of who, but a question of what. Because the answer lies in PowerShell. For those of you who want to learn more about Windows PowerShell, this TechNet article is a nice place to start.

With PowerShell we can fire off commands to our SharePoint tenant and, just to name an example, can raise the flag on a bunch of sites. So this puts you back into control and releaves you from waiting on Tech Support to pick up your ticket. Plus, you won’t have to do much scripting, because others have already done it for you. Two scripts that are particularly handy come from Mikael Svenson (https://twitter.com/mikaelsvenson).

The first script enables an admin to trigger a re-index of a site collection and all its sub sites[4]. The way the script raises the flag is by changing the search version property of the site or site collection which ensures that the site will be picked for re-indexing on the next incremental crawl. This is a major time saver in the sense that you do not have to manually trigger re-indexing on every single site

The second script allows you to raise the flag for all the user profiles in your tenant[5]. A user profile is just another content record and for it to be picked up by the crawl it needs a value change. So when you start changing user profile properties it would require a user to change something about their profile before the change is picked up. And since users do not necessarily change their profile’s very often, it might take a while before your change has reached all users. So this script is a major help in activating your change for all profiles in your tenant. Also because there is no way to raise the flag manually on a profile other than to apply a value change to that profile. And actually, this is also what the script of Mikael does. On every profile it overwrites a property value with the same value, which in the eyes of SharePoint is a value change and thus all the profiles are picked up by the next incremental crawl.

 

Summary

When working with search in SharePoint Online you have to deal with the fact that you cannot influence the crawl schedule. Just put it out of your mind and try to accept it. What you can do is make sure that all the changes that your made are picked up as soon as possible by the continuous and incremental crawls that pass by your tenant. Or, to put it in terms of our analogy, making sure that the postman is picking up your message on his very next round.

 

Disclaimer

A lot of the items discussed in this blog have been created, communicated or distributed by others as first. We certainly want to put credit where credit is due, so we tried to do our absolute best to always show the source or inventor of the trick where this was possible.

Also, the scripts mentioned in this blog should only be used and deployed by people who understand what they are doing. Never let code loose on your tenant that you do not understand yourself. This warning has nothing to do with PowerShell or these scripts in particular, but is just part of good sensible ownership for any admin.

If needed Rapid Circle can help you understand and safely deploy these scripts on your tenant and help you save time in configuring search for your SharePoint Online environment.

[1] http://www.fastcodesign.com/3034987/evidence/the-case-for-letting-employees-choose-their-own-job-titles

[2] https://support.office.com/en-us/article/Manually-request-crawling-and-re-indexing-of-a-site-a-library-or-a-list-9afa977d-39de-4321-b4ca-8c7c7e6d264e?ui=en-US&rs=en-US&ad=US

[3] https://support.office.com/en-us/article/Manually-request-crawling-and-re-indexing-of-a-site-a-library-or-a-list-9afa977d-39de-4321-b4ca-8c7c7e6d264e?ui=en-US&rs=en-US&ad=US

[4] http://www.techmikael.com/2014/02/how-to-trigger-full-re-index-in.html

[5] http://www.techmikael.com/2014/12/how-to-trigger-re-indexing-of-user.html

Watch the keynote "The Future of SharePoint" here.

Screenshot-2016-06-10-17.41.47.png

Lots of changes and innovations have been announced at the 'Future of SharePoint' conference last month. We've watched the keynote and found it very interesting. We think you might find this interesting as well. So below, find the full keynote!

Full keynote presentation "Future of SharePoint"

 

Want to know more?

Read more in-depth information in other blogs/articles by myself and my colleagues: List of #FutureOfSharePoint posts

This blog post is part of the series Future of SharePoint. More on this topic can be found athttp://rapidcircle.com.au/tag/FutureOfSharePoint/

SharePoint update in 2016: Mobile & Intelligent

On May the 4th 2016 Microsoft held their Future of SharePoint event. It presented all the new stuff which is on it's way for SharePoint and OneDrive (for Business). This post is about a break out session by Microsoft Mechanics, which highlights the new updates on its way for SharePoint in 2016. Scroll down to view the video (18:21 min), but I encourage to first read the summary. Big chances are coming. The first of which, is the update on the document libraries we reported on: New experience SharePoint Online Library.

SharePoint update

SharePoint mobile app

There will be a SharePoint which brings the SharePoint online experience to the mobile phone. It has tabs for Links, Sites and People. All driven by OfficeGraph, Search and Office 365. It's fully responsive and shows the new SharePoint sites (nextgen portals) on your mobile device. Partners should be able to modify the mobile styling to match the client. Below are 4 screenshots of the new app.

1. Links: Links list

2. Sites: showing sites you've interacted with based on OfficeGraph

3. Responsive view of a site

4. People: Search and mobile Delve profile

SharePoint App - Links
SharePoint App - Links
SharePoint App - Sites
SharePoint App - Sites
SharePoint App - Site
SharePoint App - Site
SharePoint App - People
SharePoint App - People

Sites SharePoint!

The sites app in the app launcher has it's name finally restored, it's SharePoint!. With the name update the page will be updated too. The power of OfficeGraph will be added to show an overview of sites used frequently lately but also the some extra info. Everything has been moved around to provide more real estate for the "Frequent" sites. On the left we have the option to create a new site, the Following list to display Followed sites and recently modified.

2 screenshots below;

1. Sites overview:

2. Create site: New interface to create a team or publishing site.

SharePoint overview
SharePoint overview
SharePoint - create new site
SharePoint - create new site

Nextgen Sites & Page authoring

New updates to team and publishing sites will be added to SharePoint Online. These will be fully integrated with Groups and have a new look. Navigation options are still available on the left. The biggest change that's shown is the new page authoring. Well, not really new, but the Sway interface for authoring has been added. Have a look, as it works really well in Sway, it's intuitive for users. As you may notice, adding sections is not only for documents, texts, etc. You can add Power Bi , Office Form, Quick Poll, Power App and other add-ins directly to the page.

3 screenshots below;

1. SharePoint site: New portal. with a linked group, pages slider (news) and activity.

2. Page authoring: the Sway interface to edit pages

3. Search content roll-up app: roll up content

SharePoint site
SharePoint site
SharePoint - page authoring
SharePoint - page authoring
SharePoint page
SharePoint page

New Experience for Lists

Lists will get the same new experience update as the document libraries. Still the option to place list view sections on pages, but the list itself gets a new experience.

3 screenshots below;

1. New list experience: matching the new document library experience

2. Grouped view: Grouped view in document library

3. Search: search results in document library

SharePoint List update
SharePoint List update
SharePoint - grouped view
SharePoint - grouped view
SharePoint - doc.lib
SharePoint - doc.lib

Flow

IFTTT for SharePoint. Creating automated workflows for Office 365 and 3rd party web services. This way users can create, through an UI, workflows between your favorite apps and services to get notifications, synchronize files, collect data, and more. We can automate simply tasks. We start with the trigger, like "receive a new e-mail" after which the workflow can configured. For instance adding a condition (if received from user "X") and adding actions (set category to "X", save the attachment to OneDrive for Business and sent a notification via Slack).

Screenshot:

1. Flow: creating a new flow

2. Services: connect web services to automate tasks

Flow
Flow
Flow - Services
Flow - Services

 Microsoft Mechanics episode

All coming to Office 365 in 2016

New Experience Sharepoint Online Library

The experience of SharePoint libraries is changing

Micrsoft is pushing update MC44849 which changes the user experience of libraries in SharePoint Online. They've communicated via the Message center and an article on Microsoft Office 365 Support. We've been expecting this update. For a while now we've grown fond of the new experience of the OneDrive for Business look & feel. It was to expect that this was the testing ground for the updating all SharePoint libraries to this new experience. That time is now.

Has my tenant been updated?

There are 2 ways to check whether your tenant has been updated. 1. The new experience setting has been added to the SharePoint Admin Center. Go to your SharePoint Admin Center and choose Settings. If your tenant has been updated, you should now have an extra setting: SharePoint Lists and Libraries experience.

Tenant settings new experience
Tenant settings new experience

2. When you load a library in SharePoint Online and there's a big notification banner waiting for the user. It tells "Document Libraries are getting a new look!". When this is shown, the update has been deployed.

Document Libraries are getting a new look!Document Libraries are getting a new look!
Document Libraries are getting a new look!Document Libraries are getting a new look!

Video overview

View all the changes and new functionality in this short walk through video (3:30min) about the new experience for SharePoint libraries.

Most important changes

These are the most important changes that were deployed which have a big impact on the interaction and functionality of the SharePoint library;

nieuwe ervaring - new item

- UI

If an library is displayed, the library is loaded in a screen similar to the default OneDrive for Business style (blue/white). It feels like you've left the (sub)site and are looking at the library in a different app/site. This can be confusing at first for users if they weren't informed of this change. A way to explain this, is by disconnecting the documents "source"(the library) of the site with the list view web part. The list view web part is a window to the source and the user opens the source which is presented in a different way.

- Upload

The functionality of the upload doesn't change, via drag-and-drop or through the menu option, but the ability to upload complete folders and files is new. Microsoft has learned that a lot of users were still using the classic view of OneDrive for Businees, so they could use the "Open in Explorer" function and upload complete folders. So it makes sense they have included this functionality.

- Navigation

As said, the library is displayed in it's own environment, almost looking like a separate Office 365 App/Add-in. The top bar/global navigation have been removed, together with the breadcrumb and any custom styling/master page. But the quick launch/current navigation still remain on the left. That is the user's way out/back.

- Link as contenttype

nieuwe ervaring - information pane

With the update comes the possibility for a user to add a link as an item (*.url file). It's now a default item in the "new item" menu. This means an user can add a link to a file/site/etc and place it in the library. This helps in the battle to eliminate duplicate files which can run out of sync.

- Information pane

The live preview of a file and the display of a few properties have been moved from the item's context menu to the information pane, which slides out on the right when the "I" (info) icon is clicked. This pane shows the library's properties by default. When a file is selected, it shows the live preview of the file, the properties of the file (which can be edited inline), sharing options, version history and the file properties. Very usefull.

- Spotlight/Pin

The user can now pin files and put them in the spotlight. It's like a featured image or document. If a file is pinned, a banner is added on top where the pinned files are shown as big live preview tiles. Add files there which are favorites, featured and/or important.

nieuwe ervaring - gallery view

FAQ

Have we a lost all navigation? Not quite. The quick launch / current navigation is still present on the left side. That's the ticket back to the site. Further more there is still the Office 365 app launcher.

Can the new experience be disabled? Yes. Via the SharePoint Admin Center you can disable this. go to the SharePoint Admin Center en choose Settings (direct link: https://[DOMAIN]-admin.sharepoint.com/_layouts/15/online/TenantSettings.aspx). Look for the sub header "SharePoint Lists and Libraries experience" and choose the "classic experience". This sets all document libraries to classic experience for the whole tenant.

Can I prevent the update from deploying on my tenant? No. You could delay updates by switching to a different service model. This will delay the deploy of updates with several months. But this also means that all other updates will be delayed as well. And you might not want to do so. Read more about service models on Technet: Change management for Office 365 clients.

Why is the new experience option enabled by default? Microsoft is innovating and wants you to join and make use of new features. This will at least stimulate users to take a look at first, while still giving the option (for now) to revert back to the classic experience. This stimulates usage and makes us think about a strategy on how to implement and communicate the new features.

Want to know more? Contact us at Mark.Overdijk@RapidCircle.com or Support@RapidCircle.com.

Powershell: Publishing all files in a SharePoint Online library programmatically

One of our clients build up a library of 500+ documents. After these were modified (added meta data and the content went through several rounds of corrections), we were asked to mass publish all files so the site could go live. Which leaves us with 2 options; 1. manually check-in, publish, approve all files. 2. add some CSOM & PowerShell together in a file and do it programmatically. Off course, I, Mark Overdijk, chose to persue the second option. I asked Massimo Prota to assist in getting a script ready. The first version of the script turned out rather usefull, so I added some extra features and more out-put to & interaction with the user. This latest version is generic enough so it can be re-used.

Features

- No limitation on # of files for a list/library - Added code to filter which files should be published - User will be promoted for password and confirmation - Feedback to user on screen - If a file is checked out, the script will check in before proceeding - If Content Approval is enabled for the list/library, the script will approve the file - Screen out-put will be saved to a txt file which includes the current date/time in the filename

Prerequisites Powershell

Step 1. Gather parameters

For the script to run properly, you'll need the following parameters;

.SiteUrl: This is the full URL to the (sub)site where the list is stored for which you want to publish/approve the files

.ListName: This is the Title of the list for which you want to publish/approve the files

.UserName: This is the UserName that has enough permissions to publish/approve the files

Step 2. Run PowerShell script

Start Windows Powershell as administrator.

Be sure to first set the ExecutionPolicy correctly so you are able to run scripts. Set-ExecutionPolicy Unrestricted [ENTER] Input "A" for all After the ExecutionPolicy is set, we can run the script file.

[code language="powershell"] #################################### # Script: PublishFilesSPO.ps1 # # Version: 2.0 # # Rapid Circle (c) 2016 # # by Mark Overdijk & Massimo Prota # ####################################

# Clear the screen Clear-Host

# Add Wave16 references to SharePoint client assemblies - required for CSOM Add-Type -Path "C:\Program Files\Common Files\Microsoft Shared\Web Server Extensions\16\ISAPI\Microsoft.SharePoint.Client.dll" Add-Type -Path "C:\Program Files\Common Files\Microsoft Shared\Web Server Extensions\16\ISAPI\Microsoft.SharePoint.Client.Runtime.dll"

# Parameters # Specify the subsite URL where the list/library resides $SiteUrl = "https://DOMAIN.sharepoint.com/SUBSITE" # Title of the List/Library $ListName = "TITLE" # Username with sufficient publish/approve permissions $UserName = "USER@DOMAIN.com" # User will be prompted for password

# Set Transcript file name $Now = Get-date -UFormat %Y%m%d_%H%M%S $File = "PublishFilesSPO_$Now.txt" #Start Transcript Start-Transcript -path $File | out-null

# Display the data to the user Write-Host "/// Values entered for use in script ///" -foregroundcolor cyan Write-Host "Site: " -foregroundcolor white -nonewline; Write-Host $SiteUrl -foregroundcolor green Write-Host "List name: " -foregroundcolor white -nonewline; Write-Host $ListName -foregroundcolor green Write-Host "Useraccount: " -foregroundcolor white -nonewline; Write-Host $UserName -foregroundcolor green # Prompt User for Password $SecurePassword = Read-Host -Prompt "Password" -AsSecureString Write-Host "All files in " -foregroundcolor white -nonewline; Write-Host $ListName -foregroundcolor green -nonewline; Write-Host " on site " -foregroundcolor white -nonewline; Write-Host $SiteUrl -foregroundcolor green -nonewline; Write-Host " will be published by UserName " -foregroundcolor white -nonewline; Write-Host $UserName -foregroundcolor green Write-Host " "

# Prompt to confirm Write-Host "Are these values correct? (Y/N) " -foregroundcolor yellow -nonewline; $confirmation = Read-Host

# Run script when user confirms if ($confirmation -eq 'y') {

# Bind to site collection $Context = New-Object Microsoft.SharePoint.Client.ClientContext($SiteUrl) $credentials = New-Object Microsoft.SharePoint.Client.SharePointOnlineCredentials($UserName, $SecurePassword) $Context.Credentials = $credentials

# Bind to list $list = $Context.Web.Lists.GetByTitle($ListName) # Query for All Items $query = New-Object Microsoft.SharePoint.Client.CamlQuery $query.ViewXml = " " $collListItem = $list.GetItems([Microsoft.SharePoint.Client.CamlQuery]::CreateAllItemsQuery()) $Context.Load($List) $Context.Load($collListItem) $Context.ExecuteQuery()

# Go through process for all items foreach ($ListItem in $collListItem){ # Adding spacer Write-Host " " Write-Host "/////////////////////////////////////////////////////////////" Write-Host " " # Write the Item ID, the FileName and the Modified date for each items which is will be published Write-Host "Working on file: " -foregroundcolor yellow -nonewline; Write-Host $ListItem.Id, $ListItem["FileLeafRef"], $ListItem["Modified"]

# Un-comment below "if" when you want to add a filter which files will be published # Fill out the details which files should be skipped. Example will skip all files which where modifed last < 31-jan-2015 # # if ( # $ListItem["Modified"] -lt "01/31/2015 00:00:00 AM"){ # Write-Host "This item was last modified before January 31st 2015" -foregroundcolor red # Write-Host "Skip file" -foregroundcolor red # continue # }

# Check if file is checked out by checking if the "CheckedOut By" column does not equal empty if ($ListItem["CheckoutUser"] -ne $null){ # Item is not checked out, Check in process is applied Write-Host "File: " $ListItem["FileLeafRef"] "is checked out." -ForegroundColor Cyan $listItem.File.CheckIn("Auto check-in by PowerShell script", [Microsoft.SharePoint.Client.CheckinType]::MajorCheckIn) Write-Host "- File Checked in" -ForegroundColor Green } # Publishing the file Write-Host "Publishing file:" $ListItem["FileLeafRef"] -ForegroundColor Cyan $listItem.File.Publish("Auto publish by PowerShell script") Write-Host "- File Published" -ForegroundColor Green

# Check if the file is approved by checking if the "Approval status" column does not equal "0" (= Approved) if ($List.EnableModeration -eq $true){ # if Content Approval is enabled, the file will be approved if ($ListItem["_ModerationStatus"] -ne '0'){ # File is not approved, approval process is applied Write-Host "File:" $ListItem["FileLeafRef"] "needs approval" -ForegroundColor Cyan $listItem.File.Approve("Auto approval by PowerShell script") Write-Host "- File Approved" -ForegroundColor Green } else { Write-Host "- File has already been Approved" -ForegroundColor Green } } $Context.Load($listItem) $Context.ExecuteQuery() } # Adding footer Write-Host " " Write-Host "/////////////////////////////////////////////////////////////" Write-Host " " Write-Host "Script is done" -ForegroundColor Green Write-Host "Files have been published/approved" -ForegroundColor Green Write-Host "Thank you for using PublishFilesSPO.ps1 by Rapid Circle" -foregroundcolor cyan Write-Host " " } # Stop script when user doesn't confirm else { Write-Host " " Write-Host "Script cancelled by user" -foregroundcolor red Write-Host " " } Stop-Transcript | out-null ############################## # Rapid Circle # # http://rapidcircle.com.au # ############################## [/code]

PowerShell: Terminate a workflow for all items in a list on SharePoint Online

This is a follow up on our previous post "PowerShell: Start a workflow for all items in a list on SharePoint Online". As it's great that now there's a script available to start a workflow for all items, it would also be great to have the ability to stop all workflows if necessary. So I, Mark Overdijk, got to work again with Massimo Prota to get this script in place. The script is very similar to the StartWorkflow Powershell script, but the difference is that we don't retrieve the workflow through WorkflowAssociations but we have to use WorkflowInstances.

Prerequisites Powershell

Step 1. Gather parameters

For the script to run properly, you'll need the following parameters;

.SiteUrl: This is the full URL to the (sub)site where the list is stored for which you want to run the workflow

.ListName: This is the Title of the list for which you want to run the workflow

.UserName: This is the UserName that has enough permissions to run the workflow

Step 2. run PowerShell script

Start Windows Powershell as administrator.

Be sure to first set the ExecutionPolicy correctly so you are able to run scripts. Set-ExecutionPolicy Unrestricted [ENTER] Input "A" for all After the ExecutionPolicy is set, we can run the script file.

Copy/Paste the code below in a txt file and save as an *.ps1 file (in this example "StopWorkflow.ps1"). Fill out the parameters with the gathered information and run the script.

PowerShell stop workflow
PowerShell stop workflow

[code language="powershell"] # Add Wave16 references to SharePoint client assemblies and authenticate to Office 365 site - required for CSOM Add-Type -Path "C:\Program Files\Common Files\Microsoft Shared\Web Server Extensions\16\ISAPI\Microsoft.SharePoint.Client.dll" Add-Type -Path "C:\Program Files\Common Files\Microsoft Shared\Web Server Extensions\16\ISAPI\Microsoft.SharePoint.Client.Runtime.dll" Add-Type -Path "C:\Program Files\Common Files\Microsoft Shared\Web Server Extensions\16\ISAPI\Microsoft.SharePoint.Client.WorkflowServices.dll"

# Specify tenant admin and site URL $SiteUrl = "https://[TENANT].sharepoint.com/" $ListName = "[TITLE OF THE LIST]" $UserName = "[USERNAME]" $SecurePassword = Read-Host -Prompt "Enter password" -AsSecureString

# Bind to site collection $ClientContext = New-Object Microsoft.SharePoint.Client.ClientContext($SiteUrl) $credentials = New-Object Microsoft.SharePoint.Client.SharePointOnlineCredentials($UserName, $SecurePassword) $ClientContext.Credentials = $credentials $ClientContext.ExecuteQuery()

# Get List $List = $ClientContext.Web.Lists.GetByTitle($ListName)

$ClientContext.Load($List) $ClientContext.ExecuteQuery()

$ListItems = $List.GetItems([Microsoft.SharePoint.Client.CamlQuery]::CreateAllItemsQuery()) $ClientContext.Load($ListItems) $ClientContext.ExecuteQuery()

# Create WorkflowServicesManager instance $WorkflowServicesManager = New-Object Microsoft.SharePoint.Client.WorkflowServices.WorkflowServicesManager($ClientContext, $ClientContext.Web)

# Connect to WorkflowSubscriptionService $WorkflowSubscriptionService = $WorkflowServicesManager.GetWorkflowSubscriptionService()

# Connect WorkflowInstanceService instance $WorkflowInstanceService = $WorkflowServicesManager.GetWorkflowInstanceService()

$ClientContext.Load($WorkflowServicesManager) $ClientContext.Load($WorkflowSubscriptionService) $ClientContext.Load($WorkflowInstanceService) $ClientContext.ExecuteQuery()

# Get WorkflowAssociations with List $WorkflowAssociations = $WorkflowSubscriptionService.EnumerateSubscriptionsByList($List.Id) $ClientContext.Load($WorkflowAssociations) $ClientContext.ExecuteQuery()

# Prepare Terminate Workflow Payload $EmptyObject = New-Object System.Object $Dict = New-Object 'System.Collections.Generic.Dictionary[System.String,System.Object]'

# Loop Terminate Workflow For ($j=0; $j -lt $ListItems.Count; $j++){

$msg = [string]::Format("Killing workflows {0} on ListItemID {1}", $WorkflowAssociations[0].Name, $ListItems[$j].Id) Write-Host $msg

$itemWfInstances = $WorkflowInstanceService.EnumerateInstancesForListItem($List.Id, $ListItems[$j].Id) $ClientContext.Load($itemWfInstances) $ClientContext.ExecuteQuery() for ($k=0;$k -lt $itemWfInstances.Count;$k++) { try { $WorkflowInstanceService.TerminateWorkflow($itemWfInstances[$k]) $msg = "Worfklow terminated on " + $ListItems[$j].Id $ClientContext.ExecuteQuery() } catch { $msg = "Error terminating workflow on " + $ListItems[$j].Id + " Details: $_" }

Write-Host $msg } } [/code]

PowerShell: Start a workflow for all items in a list on SharePoint Online

For one of our Office 365 clients (mix of E1 and E3 licences) we created a workflow which will check the status of an item and, depending on this status, sends out e-mails and updates other columns. As the list was already in use, it was necessary to start the workflow for all present items. But to start the workflow manually for all 477  items, was not preferable. So I, Mark Overdijk, asked Massimo Prota to help me on the quest to see if it would be possible to to do it via PowerShell. As there are no PowerShell commands available for SharePoint Online to access the workflow instance, we searched for CSOM solutions. We came across this script on github. Thanks to Azam-A we had a base script to work from. What we changed/added were the following;

  • Referenced the new wave16 components as Office 365 is already on wave16.
  • Added feedback in the script when it runs. It'll show for each item the item ID for which the script is starting the workflow.
  • For obvious security reasons we're not storing the user's Admin password as plain text, but prompt for the password.

Prerequisites Powershell

Step 1. Gather required parameters

For the script to run properly, you'll need the following parameters;

.SiteUrl: This is the full URL to the (sub)site where the list is stored for which you want to run the workflow

.ListName: This is the Title of the list for which you want to run the workflow

.UserName: This is the UserName that has enough permissions to run the workflow

 

Step 2. Run PowerShell script

Start Windows Powershell as administrator.

Be sure to first set the ExecutionPolicy correctly so you are able to run scripts. Set-ExecutionPolicy Unrestricted [ENTER] Input "A" for all After the ExecutionPolicy is set, we can run the script file.

Copy/Paste the code below in a txt file and save as an *.ps1 file (in this example "StartWorkflow.ps1"). Fill out the parameters with the gathered information and run the script.

StartWorkflow1 screenshot in  Powershell

[code language="powershell"] # Add Wave16 references to SharePoint client assemblies and authenticate to Office 365 site - required for CSOM Add-Type -Path (Resolve-Path "$env:CommonProgramFiles\Microsoft Shared\Web Server Extensions\16\ISAPI\Microsoft.SharePoint.Client.dll") Add-Type -Path (Resolve-Path "$env:CommonProgramFiles\Microsoft Shared\Web Server Extensions\16\ISAPI\Microsoft.SharePoint.Client.Runtime.dll") Add-Type -Path (Resolve-Path "$env:CommonProgramFiles\Microsoft Shared\Web Server Extensions\16\ISAPI\Microsoft.SharePoint.Client.WorkflowServices.dll")

# Specify tenant admin and site URL $SiteUrl = "https://[TENANT].sharepoint.com/" $ListName = "[TITLE OF THE LIST]" $UserName = "[USERNAME]" $SecurePassword = Read-Host -Prompt "Enter password" -AsSecureString

# Connect to site $ClientContext = New-Object Microsoft.SharePoint.Client.ClientContext($SiteUrl) $credentials = New-Object Microsoft.SharePoint.Client.SharePointOnlineCredentials($UserName, $SecurePassword) $ClientContext.Credentials = $credentials $ClientContext.ExecuteQuery()

# Get List and List Items $List = $ClientContext.Web.Lists.GetByTitle($ListName) $ListItems = $List.GetItems([Microsoft.SharePoint.Client.CamlQuery]::CreateAllItemsQuery()) $ClientContext.Load($List) $ClientContext.Load($ListItems) $ClientContext.ExecuteQuery()

# Retrieve WorkflowService related objects $WorkflowServicesManager = New-Object Microsoft.SharePoint.Client.WorkflowServices.WorkflowServicesManager($ClientContext, $ClientContext.Web) $WorkflowSubscriptionService = $WorkflowServicesManager.GetWorkflowSubscriptionService() $WorkflowInstanceService = $WorkflowServicesManager.GetWorkflowInstanceService() $ClientContext.Load($WorkflowServicesManager) $ClientContext.Load($WorkflowSubscriptionService) $ClientContext.Load($WorkflowInstanceService) $ClientContext.ExecuteQuery() # Get WorkflowAssociations with List $WorkflowAssociations = $WorkflowSubscriptionService.EnumerateSubscriptionsByList($List.Id) $ClientContext.Load($WorkflowAssociations) $ClientContext.ExecuteQuery()

# Prepare Start Workflow Payload $Dict = New-Object 'System.Collections.Generic.Dictionary[System.String,System.Object]'

# Loop List Items to Start Workflow For ($j=0; $j -lt $ListItems.Count; $j++){ $msg = [string]::Format("Starting workflow {0}, on ListItemId {1}", $WorkflowAssociations[0].Name, $ListItems[$j].Id) Write-Host $msg #Start Workflow on List Item $Action = $WorkflowInstanceService.StartWorkflowOnListItem($WorkflowAssociations[0], $ListItems[$j].Id, $Dict) $ClientContext.ExecuteQuery() }[/code]

If, for some reason, you want to stop/terminate all workflows, check this blogpost: PowerShell: Terminate a workflow for all items in a list on SharePoint Online

Walkthrough: Add Geolocation column to your list in Office 365

A while ago a client (with an Office 365 E3 subscription) came to us with the wish to create a map to plot locations of external contractors on. My first thoughts, as an Office 365 consultant, went towards using the tools at hand. SharePoint 2013/Online has a Geolocation column type and the list view type "Map view". The client agreed to use this feature and I went about setting up the solution. So I posed the self-fulfilling prophecy: "How hard can it be?"...

As the list with the data was already in place, I was neither keen on letting a developer create a solution which either creates a new list with the column in it (and me migrating data) nor writing a solution which adds the column programmatically once. I wanted to add the column directly through (a reusable) script and went on to do my desk research. This ended up taking way too much time as almost all information found…

  • ...were solutions for SharePoint 2013 on premise,
  • ...were articles on the end-result,
  • ...posted failing scripts,
  • ...did not offer information on the Bing Maps key,
  • ...did not offer guides/information specifically for Office 365/SharePoint Online scenario's.

Something as simple as "what to use as the Bing Maps application URL for an Office 365 tenant?" was not to be found.

It took a while, but when I finally got the settings right for a Bing Maps key and a working script, I decided on 2 things;

  1. Create a generic script, because as a consultant I'll want to use this script more than once for multiple tenants.
  2. Write a blog post as a definitive guide to add the geolocation column type in Office 365/SharePoint Online as a resource for the community

Scenario

For the walkthrough I'm using the following scenario; As a global admin for the tenant https://yourcompany.sharepoint.com, I'm adding the geolocation column type to the list "Contact" on the sub site https://yourcompany.sharepoint.com/sites/sales and naming the column "Office".

Step 1. Get a Bing Maps Key

Go to Bings Maps Dev Center; https://www.bingmapsportal.com/

Log in with your Live account (@live.com, @outlook.com, etcetera) or create one to gain access.

Go to My account > Create or view keys

bingmapsdev1
bingmapsdev1

To create a new API key follow the "Click here to create a new key" hyperlink

bingmapsdev2
bingmapsdev2
bingmapsdev3
bingmapsdev3

Fill out the form to create your API key

. Application name: The name you would like to use for your key. It helps you to identify the key in your overview

. Application URL: The URL of your root SharePoint portal (https://tenant.sharepoint.com)

. Key type (Trial/Basic): Choose whether you're using the key for 1) a test site (max 10,000 calls p/mth and max 90-day trial period) or 2) a live site (free for max 125,000 calls p/yr) (more info here)

. Application type: What is the application? App, site,for non-profit use, etc.

In this scenario, the admin fills it out;

  • Application name: Sales Office
  • Application URL: https://yourcompany.sharepoint.com
  • Key type: Basic
  • Application Type: Public Website

After you click Create and the Captcha was filled out correctly, the page refreshes and displays your new key below. You'll receive a 64 character key.

bingmapsdev4
bingmapsdev4

Step 2. Gather required information

For the script to run properly, you'll need the following information;

. Site URL: URL to the site where the list is.

. Login account: at least admin permission as you're changing list settings.

. List Name: name of the list to add the geolocation column type.

. Column Name: title of the geolocation column.

. Bing Maps Key: to register the app and remove the notification in map view.

In this example, the admin has gathered the following info;

  • Site URL: https://yourcompany.sharepoint.com/sites/sales
  • Creds: Admin@YourCompany.onmicrosoft.com
  • List Name: Contact
  • Column Name: Office
  • Bing Maps Key: [PASTE KEY HERE]

Now we can run the script.

Step 3. Run script

Start SharePoint Online Management Shell as administrator

If you don't have SharePoint Online Management Shell, you can download it @ Microsoft Download Center

set-executionpolicy Unrestricted Clear-Host [void][System.Reflection.Assembly]::LoadWithPartialName('Microsoft.VisualBasic') <# Get User input #> $SiteURL = [Microsoft.VisualBasic.Interaction]::InputBox("Enter Site URL, example: https://yourtenant.sharepoint.com/sites/yoursite", "URL", "") $Login = [Microsoft.VisualBasic.Interaction]::InputBox("Office 365 Username, example: youradmin@yourtenant.onmicrosoft.com", "Username", "") $ListName = [Microsoft.VisualBasic.Interaction]::InputBox("List name to add Geolocation column", "ListName", "") $ColumnName = [Microsoft.VisualBasic.Interaction]::InputBox("Column name for the Geolocation column", "ColumnName", "") $BingMapsKey = [Microsoft.VisualBasic.Interaction]::InputBox("Bing Maps key", "Key", "") <# Show results #> Write-Host "/// Values entered for use in script ///" -foregroundcolor magenta Write-Host "Site: " -foregroundcolor white -nonewline; Write-Host $SiteURL -foregroundcolor green Write-Host "Useraccount: " -foregroundcolor white -nonewline; Write-Host $Login -foregroundcolor green Write-Host "List name: " -foregroundcolor white -nonewline; Write-Host $ListName -foregroundcolor green Write-Host "Geolocation column name: " -foregroundcolor white -nonewline; Write-Host $ColumnName -foregroundcolor green Write-Host "Bing Maps key: " -foregroundcolor white -nonewline; Write-Host $BingMapsKey -foregroundcolor green Write-Host " " <# Confirm before proceed #> Write-Host "Are these values correct? (Y/N) " -foregroundcolor yellow -nonewline; $confirmation = Read-Host if ($confirmation -eq 'y') { $WebUrl = $SiteURL $EmailAddress = $Login $Context = New-Object Microsoft.SharePoint.Client.ClientContext($WebUrl) $Credentials = Get-Credential -UserName $EmailAddress -Message "Please enter your Office 365 Password" $Context.Credentials = New-Object Microsoft.SharePoint.Client.SharePointOnlineCredentials($EmailAddress,$Credentials.Password) $List = $Context.Web.Lists.GetByTitle("$ListName") $FieldXml = "<Field Type='Geolocation' DisplayName='$ColumnName'/>" $Option=[Microsoft.SharePoint.Client.AddFieldOptions]::AddFieldToDefaultView $List.Fields.AddFieldAsXml($fieldxml,$true,$option) $Context.Load($list) $Context.ExecuteQuery() $web = $Context.Web $web.AllProperties["BING_MAPS_KEY"] = $BingMapsKey $web.Update() $Context.ExecuteQuery() $Context.Dispose() Write-Host " " Write-Host "Done!" -foregroundcolor green Write-Host " " } else { Write-Host " " Write-Host "Script cancelled" -foregroundcolor red Write-Host " " }
The actual programming part in the script I modified from the script posted ina blog post by Albert Hoitingh. I wanted to remove the hardcoded values from the code, so the script can be run based on user inputs. So I added the interface (input boxes, confirmation, write-hosts), replaced the hard coded values and added comments.

When you run the script, PowerShell will ask the user to input the information we gathered in Step 2.

geoscript1
geoscript1
geoscript2
geoscript2
geoscript3
geoscript3
geoscript4
geoscript4
geoscript5
geoscript5

After the last values have been entered, the admin will see a confirmation screen where the values can be review and confirmed (if the input is incorrect, the script can be cancelled by entering "N" to not proceed - screenshot).

geoscript6
geoscript6

After confirmation, the admin will be prompted to enter the password.

geoscript7
geoscript7

If everything was filled out correctly, the script will run and returns with the "Done!" notification upon completion.

geoscript8
geoscript8

Return to your SharePoint online list and you'll notice when creating a new view for your list "Contact", you gained the option Map View. When checking the list settings, the column "Office" has been added of the type Geolocation.

mapview1
mapview1

Are you missing information, do you want me to clarify anything, do you want to post a conversation starter or do you just want to say thanks? Leave a comment.

Get User Information from Client OM (JS)

When I was exploring the possibilities to read from a User Profile in a SharePoint Online environment I realized there are lots of properties that are being synced back to the UserInfoList, but only when they are filled in the User Profile.

User Info List

We know by now it's impossible from a Sandboxed Solution (or Client Side OM Solution) to access a Users Profile directly as you would in a On-Premise environment. Our scope is the SiteCollection that runs our custom code. In that scope there is one important resource we can use to get information about a User, the UserInfoList. The list contains some Properties that are populated by the SharePoint Profile. So somehow this is synced.

Synced Properties

The synchronization process is done (On-Premise) by a TimerJob called: "User Profile Service Application - User Profile to SharePoint Full Synchronization". This TimerJob normally runs "Hourly". I didn't test this out but my guess is that this will be the case with SharePoint Online too. The important part being that there is a delay in this procedure. That being said, which properties of the User Profile are being synced by this TimerJob?

Property StaticName (Client OM) Returns
Account name Name domainierselca or claim
User name UserName ierselca
Name (DisplayName) Title Cas van Iersel
First name FirstName Cas
Last name LastName van Iersel
Title JobTitle SP Consultant
Email EMail cas@rapidcircle.com
Work phone WorkPhone 0123456789
Mobile phone MobilePhone 0123456789
Picture Picture http://linktoimage *
Department Department R&D
About me Notes Inner HTML of About field as a string
Web site WebSite http://mywebsite.com *
SIP SipAddress sip format string
Office Office Rapid Circle
Ask me about SPSResponsibility SharePoint, C#, ASP
If User Is Site Admin IsSiteAdmin true | false

* You'll need to do .get_url() on the return object

This is an example on how to get the information based on the UserID in the Context of the current web (You can obtain this from the SPUser object or use javascript to get from a people link "person.aspx?ID={UserID}":

//Get the Context, Web and SiteUserInfoList for User Info var clientContext = SP.ClientContext.get_current(); var web = clientContext.get_web(); var collListItem = web.get_siteUserInfoList().getItemById(userID);

//Load the objects async clientContext.load(collListItem); clientContext.executeQueryAsync( Function.createDelegate(this, function(){ //Read all properties from the Item var userName = collListItem.get_item('Name'); var website = collListItem.get_item('WebSite').get_url(); }), Function.createDelegate(this, function(){ //On Fail }));

Note: When the field is empty in your profile the get_item() Method will return null on the property.

Profile Picture

On property that can be slightly inconvenient is the Profile Picture. The URL is stored in the UserInfoList so that will be returned perfectly. But with SharePoint Online the Picture is most definitely stored in the MySite Host webapp. This means the user is challenged for credentials when this URL is used in an <img /> tag for instance.

Summary

There is a lot of User Information we can work with using the Client OM. It's very easy to obtain the information using Javascript and to do cool stuff with it. Just keep in mind that the syncing process can take some time and no custom properties are being synced.