Modern Retention using SharePoint Metadata

Reading Time: 5 minutes

[Updated June 2019]

Retention labels in Office 365 are configured in the Security and Compliance Center. You can manually set a label or use the auto-apply feature to automatically set it based on a condition. The condition can be based on one of two things:

  • sensitive information type
  • keyword query

If you’re coming from the legacy Information Management Policy world where you could set retention based on a piece of metadata or a content type on a document, you are likely looking for similar capability in the new world of Retention labels. Up until this point in time, this wasn’t possible, however this capability is now released to Office 365 tenants with an E5 license!

License explanation from Microsoft:

The capability to apply labels automatically requires an Office 365 Enterprise E5 license for each user who has permissions to edit content that’s been automatically labeled in a site or mailbox. Users who simply have read-only access do not require a license.

The keyword query uses KQL syntax. In this syntax, you can search for any managed property in SharePoint that is queryable. This requires an understanding of how SharePoint metadata translates into a managed property in the search schema. Once you understand this, it opens up a world of possibilities for automatically setting retention based on a SharePoint metadata value. Examples of where this might be useful:

  • apply a Budget retention label against all budgets when their status is ‘approved’
  • apply a Contract retention label when the contract status is ‘Expired’

Follow below for the steps to implement the second example above to apply a Contract retention label based on a custom SharePoint column on a custom Content Type.


Create Site and Information Architecture to hold Contracts

For this example, I’ll create a new modern SharePoint Communication site called Contract Central. Within this site, I’ll define the Information Architecture (IA) to store all contracts. This will include a Content type called Contract with the following site columns:

  • Contract status – choice values: Pending, Active, Expired
  • Contract effective date – date
  • Contract expiry date – date

Create a new document library called Contracts. In this library, include the custom Contract content type from above.

Add some contract documents to the Contracts library and fill in the metadata. Ensure some of them are set to each of the contract statuses to demonstrate the auto-apply feature. In the image below, I’ve added 6 contract documents, all of the Contract content type, 3 of which are set to Expired.

ContractLibraryBeforeAutoApply


Let Search do its Magic!

Wait for search to index the content added above. During this process, crawled and managed properties are generated in the search schema and, when complete, I’ll leverage the Contract Status managed property in this solution.

To build a query using managed properties, they must be queryable (have the Query property set) as shown below.

The first property to query is the content type. I only want to apply the Contracts label to documents that are a Contract content type. To do this, I’ll query the ContentType managed property (queryable property is automatically set – circled):

ContentTypeManagedPropertyQueryable

Current auto-apply functionality requires a mapping from the choice column crawled property, in our case ows_q_CHCS_ContractStatus, to one of the pre-configured RefinableString## managed properties, in this case RefinableString10. Although the choice column managed property, ContractStatusOWSCHCS,  appears Queryable when you edit the property settings, as of the time of this writing, it cannot be used in the KQL to auto-apply a label.

This slideshow requires JavaScript.

Once this mapping is done in the search schema, it’s time to define the retention label.

Tip: to test your query before proceeding to the next step, enter it in a search box in SharePoint. You should get back the correct search results:

ContentType:Contract AND RefinableString10:Expired


Create Retention Label

Retention labels are added in the Security & Compliance Center under the Classifications section. I created a new retention label called Contract with these settings in accordance with the retention requirements for my organization. Retention will commence when the document is labeled since we want to only label it when a contract is expired and will be automatically deleted 3 years after that:

ContractLabelDefinition


Create Label Policy

To auto-apply the Contract label from above, select the Auto-apply a label button  from the label definition pane.

AutoApplyLabel

This will prompt you to fill out some settings to auto-apply the label. For the Contract label, we want to set it based on specific words or phrases, so select the second option:

TypeOfContent

In the Keyword query editor box that appears, enter your KQL query to include the query to find all Contracts that have expired. To do this, we query on a Content type of Contract and a Contract status of Expired. (image)

KQL Joint ConditionNext, name the policy and specify the Contract Central site as the publish location. The auto-apply will execute against that location only. When you save the policy, the status will move from Pending to Success once fully deployed.


Wait for the auto-apply

Once you save the Retention Policy with this updated query, the auto-apply can take up to 7 days to apply the label to all content across your tenant in the locations you specified. For this test, it took 3 days. As you can see from the results, the Contract retention label was only applied to documents where the Content type is Contract and the status is Expired:

ContractLibraryAfterAutoApply2


Where to from here?

This capability brings a lot of flexibility to the retention label picture for SharePoint. It can be used to automate some of the retention scenarios in your organization making you less reliant on end-users to manually label a document. Always a good thing.

Thanks for reading!

-JCK

9 comments

  1. Excellent and I look forward to your next post. Is it possible to update the keyword policy settings, say if there is a change to the IA / managed properties, and will this deploy?

  2. Most handy would be a Microsoft solution using a similar kind of construction as Flow: pick your query blocks (worded in nice, easy layman’s language), and add your details to each block…

  3. Thanks for the excellent post. I’m wondering if you have attempted this lately. I followed your post and have success with running the KQL in search. Unfortunately, you can no longer save the Auto-Apply policy as it generates an error on the custom column as being invalid.

    1. Hi Brad,
      This is now supported functionality. You need to ensure the custom column managed property is queryable. If not, map the managed property to a “Refinable” pre-configured managed property and use that in your query instead. You can use ContentType as-is.
      I have done this just last week and it worked. You can also do a joint condition in the query from my testing. I.e. ContentType:Contract AND RefinableString##:Technology

      -JCK

  4. Hi Joanne, I have an issue where I’m unable to deploy a retention policy (and therefore label) to our SharePoint environment. When trying to publish the label (I’m only trying to publish it to a single SharePoint site at present, no other locations) I get the following error message within the Compliance Centre – ‘Policy cannot be deployed to the content source due to a temporary Office 365 datacenter issue’. Have you (or any of your blog followers) come across this issue? A search for this error message brings back very little online other than in relation to MDM (which I believe we do have configured on our tenancy but not sure if this is likely to conflict with the publishing of retention labels)? Any advice would be appreciated. Thanks, Kieran.

    1. Hi Karen, I’ve never encountered or heard of this error. Since the error message says “temporary”, I’m assuming you have tried again? If you’re still getting the error and you’ve tried basic troubleshooting (try publishing to different site), I would open a ticket with Microsoft to resolve the issue.
      -JCK

      1. I have run across a similar issue. I suggest you check the O365 Health Service to ensure that all services in your region are running without degradation. My issue was resolved in about 6 hours when the O365 Health was restored to normal.

Leave a Reply to Keith Cancel reply

This site uses Akismet to reduce spam. Learn how your comment data is processed.