Have you heard about Adaptive Scopes? It’s a new Microsoft Information Governance and Records Management (IG/RM) feature now in Public Preview (refer to Roadmap ID 70578). I’m really excited about this new feature because I think it addresses many of the challenging retention scenarios I see with customers looking to apply IG/RM controls at scale across their tenant.
[Updated October 18, 2021]
Adaptive scopes will help organizations gain compliance across their environment in a flexible, scalable, automated, and targeted way. Click To Tweet
Microsoft’s recent webinar provides walk-thru demos of the new Adaptive Scope feature for several common use-case scenarios. Check it out: mipc.eventbuilder.com/event/45703
A few key things about adaptive scopes to understand:
- They are scopes, not policies
- They are referenced when publishing either retention policies or retention label policies
- There will now be 2 types of scopes:
- Static (what we’ve had up until now)
- Adaptive (new)
- There are 3 types of adaptive scopes:
- User | based on Azure AD attributes
- Site | based on SharePoint site properties
- Microsoft 365 Group | based on Azure AD attributes
Check out my Collection of posts to demonstrate how to use Adaptive Scopes to address common retention requirements I see in the field. Each post in the collection will demonstrate how to use an Adaptive Scope in a slightly different way.
This is the first post in the collection.
The Retention Requirement ask for this post:
“Retain all content on Project sites for 10 years once the project is complete. Content does not need to go thru a disposition review once the retention period has been met.”
To accomplish this, I’ll combine an Adaptive scope with a Retention Policy set to retain for 10 years then delete. The Retention Policy will automatically include completed Project SharePoint sites without having to manually add each Project Site URL to the Retention Policy once the project is complete.
Tip: the implementation of this solution will be easier if you have some SharePoint Admin/Search experience.
High-level steps to make this happen:
- The SharePoint project site setup
- The SharePoint search index
- The Adaptive scope
- The Retention Policy
- The end result
Let’s dig in.
Step 1: The SharePoint Project Site Setup
In this example, I’m using automation in my environment to provision Project sites. There are many ways to auto-provision a site. Please refer to another post of mine, Auto-provisioning SharePoint Sites and Teams: A blog post list, where I reference several ways of doing it.
In this blog post example, a Project Requests SharePoint list is central to the process. A new item is added by a Project Manager to kick off the creation of a new Project which means a new Project SharePoint site! Automating the provisioning process is required to standardize the information architecture across all Project sites, assign ownership, join each to a Project Hub, and set the Project site’s sensitivity to control the corresponding privacy, external, and device access controls.
In addition to this, I’ll be adding a status to the Project site to indicate the overall status of the Project… either Active or Complete. The status will be set to Active when the site is initially provisioned and Complete when the Project is done. I do this because I only want completed Project sites to be included in the Retention policy.
Wondering how to set a property at the site level? You add a value in the site property bag during the site provisioning process. By setting a property at the site level, you can control the overall status of the entire site. In this way, once the project is complete, the Project Requests SharePoint list item can be updated by the Project Manager to set the project’s status to complete which will kick off a process to update the status property of the Project site’s property bag to Complete.
Updating the property bag of a SharePoint site is relatively straight forward once you know a few things. The site property bag is a dictionary of key-value pairs to define properties of the site… we want to add a few custom ones! To be clear, the site property bag can be used for far more than just Adaptive scopes; however for this post, that’s what I’m using them for. 🙂
I used PnP PowerShell in my provisioning solution to connect to the Project site and set 2 properties: (in addition to enabling scripts prior to setting the property bag values, you should also disable scripts when you’re done!)
- Property: customSiteType set to “ProjectSite”. This property is used to identify Project sites across the tenant. This property will be used in the Adaptive scope condition to only include Project sites.
- Property: customSiteStatus set to “Active” when the site is provisioned; “Complete” when the project is complete. This property will be used in the Adaptive scope condition to determine if the Project site should be included in the Retention policy. We only want those with a status of Complete to be included.
To demonstrate, I provisioned 3 project sites and set the 2 custom properties shown above on each site as follows:
- Sample Project A: customSiteType = ProjectSite, customSiteStatus = Complete
- Sample Project B: customSiteType = ProjectSite, customSiteStatus = Active
- Sample Project C: customSiteType = ProjectSite, customSiteStatus = Complete
With this setup, 2 of the 3 Project sites are Complete and will be the only 2 sites included in the adaptive scope.
Step 2: The SharePoint Search index
The hard part is done! Setting up the search index is the same as what you’re used to when defining managed properties for custom SharePoint columns, except this time we’re using crawled properties generated from custom site property bag properties instead of SharePoint columns.
Browsing to the tenant-level search schema, I see 2 crawled properties have been automatically generated that map directly to the 2 custom property bag properties: customSiteType and customSiteStatus.
I need to map each of these to a pre-built RefinableString managed property (00-99 only) in the search schema so they can added to the Adaptive scope condition. I pick RefinableString50 and RefinableString51 respectively.
Once the crawled properties are mapped to the managed properties, I initiated a re-index of the Project sites. This is only required in the setup step so I can verify the search results prior to using the RefinableString managed properties in the Adaptive scope condition.
I recommend validating the KQL query before using it in your Adaptive scope condition. For this particular query, it will work by entering it in either the Microsoft Search box (shown below) or using Content Search in the Compliance Center. As a general rule of thumb, ensure the query will return expected results from Content Search as this more closely aligns with the search used by Compliance features.
In both locations, this is the query to validate it meets our conditions:
RefinableString50 = ProjectSite AND RefinableString51 = Complete
Microsoft Search method
Content Search method
…in both cases, it correctly returns the 2 completed Project sites so I’m confident moving forward with my conditions when defining the adaptive scope.
Step 3: The Adaptive Scope
You’ll see the Adaptive scopes tab in 2 places in the Compliance Center: within Information governance and within Records management. This is because you can define an adaptive scope for either a retention policy or a retention label policy. In our example, we’re applying a retention policy so I’ll go to Information governance… Adaptive scopes:
I’ll create a scope called Project Site Retention Scope:
And I’ll create a SharePoint sites scope because I want to use my site properties as the condition:
I’ll add my 2 RefinableString managed property conditions (referenced in Step 2) to only include Completed Project sites in this scope:
The Adaptive scope takes a bit of time to process in the backend (up to 24 hours) to bring in the relevant locations that match the query. In my example, within a day, the 2 sites were included in the scope when viewing the scope details; however, please know the Adaptive scope can be used immediately when publishing a retention policy or label policy before the sites show in the below list.
Step 4: The Retention Policy
If you recall, this is the retention requirement we’re implementing:
“Automatically apply a 10 year retention policy for all completed Project SharePoint sites”
I’ll create a new retention policy with a type of Adaptive:
I’ll add the Adaptive scope created in the previous step, Project Site Retention Scope to the policy and ensure only ‘SharePoint sites’ is selected as a location because I know that Project Sites will never be OneDrive accounts:
… and last, but not least, I set the retention settings for this retention policy:
When I look at the policy summary, I can see it is scoped to my Adaptive scope:
Step 5: The end result
Once the retention policy is published and the adaptive scope reflects the correct scoped list of sites, the actions configured by the retention policy begin. In this example, the 2 completed project sites now have the retention policy applied to them and so the “10 year retention” setting applies.
If I delete any content/change any content on the project sites, it will be preserved in the Preservation Hold library on the site and, after 10 years, the content will be automatically deleted. This will follow normal recycle bin processing.
There is tremendous value in this model to scope retention in a flexible, scalable, and automated way. As I come across other use cases while working with customers, I’ll share the setup and configuration in the collection of posts.
If you have an example you’d like me to cover, share in the comments!
Thanks for reading.
Awesome breakdown of adaptive scopes! I always look forward to your blogs, they definitely help further explain some of Microsoft’s vague documentation 😉 Looking forward to giving Adaptive Scopes a spin when they’re released to GA!
Thanks Brian! Once Adaptive scopes are officially GA, there will be in depth documentation from Microsoft as well. At that time, I’ll also blog about some more use cases.
One problem with “include” scoping was the need to exclude from other policies that may also apply. This might be a silly question having not looked into it, but does this obviate the need for that? Or would we need mirrored adaptive scoping? 🙂
Hi Todd! I believe the principles of retention will take care of that. Special mention: once a retention label has been applied to an item based on an adaptive scope, it will remain on the item even if the scope changes.
Yeah – I get that part – I was thinking of this scenario, which was challenging before from an inclusion AND exclusion perspective:
Previous to adaptive scopes:
Scenario (medium complexity): 5,000 user org, need shorter retention for subset of users, account-specific targets (EXO only; Chats require separate policies; OneDrive not included in scenario due to intractability of non-global policy once get into the 1000s of users).
Group 1 – 1,100 users
• Policy A – 3 years retention for 1,000 users (inclusion)
• Policy B – 3 years retention for 1,001+ (inclusion)
Option 1: Global – policy C – 5 years retention, no exclusions – “fallback policy”
Result: all users will be subject to C (due to longer retention being applicable to them) and thus Policies A and B won’t work – see next option.
• Global – policy C – 5 years retention, with exclusions for members included in A
• Global – policy D – 5 years retention, with exclusions for members included in B
Result: users subject to A and B will have 3 years retention, everyone else 5.
o Even in this intermediate scenario, it demonstrates how this can get quite complex, quickly (4 policies required for this inclusion/exclusion scenario).
o New users added to Group 1 later will need to be added to Policy B (inclusions) and D (exclusions)
o Users in A or B that leave will need to be dealt with
o Need to break out for different workloads, so this gets even more complicated.
o Results in 4 policies for EXO, 4 for chats, and if OneDrive was included in this mix, 24 more.
Net net: could end up with 32 policies with this fairly simple scenario if you want to cover all the account targets.
Post adaptive scopes (was looking for insight on whether this is correct):
1) The need for policies A and B can be addressed through adaptive scope, but Option 1 result will still be the same, since still has longer retention in Policy C.
2) Option 2 – could the global policy have mirrored exclusions for those not subject to the adaptive scope inclusion, or do you still have to manually manage exclusions?
Great question! Based on my understanding of the scenario you described… you will need adaptive scopes to align with both the inclusion and the exclusion of your subset of users…They will basically be the inverse of each other. Then define your policies to refer to either one or the other.
Thanks for your great tutorial Joanne ! I was looking for a long time to create retention policies based on different countries, maybe this is the answer . What license would be required for Adaptive Scope Retention Policies ?
Hi Joanne, these are very nice examples as to where adaptive scopes make sense.
I have taken over some of your content into my presentation.
Hi Bernd, that’s awesome! Please give credit to my blog post for your presentation. Thanks!
Hi Joanne, Thank you so much for this examples. I created a script to set the managed property on multiple Sharepoint sites, but I was wondering about the reindexing part. I have not found a cmdlet to this and doing it manually is not an option. Do you have more information about can I automatize reindexation?
Thank you so much in advance