I think we all agree automating as much retention as possible is a good thing. The less we have to rely on information workers to manually apply a retention label, the better. The information architecture you’ve diligently defined in your tenant can now be leveraged using auto-apply conditions to automatically set an Office 365 retention label.
[UPDATED October 3, 2019] Added a section on doing an auto-apply against a managed metadata column type.
Microsoft has also announced machine learning classifiers to help with the growing amount of corporate “dark data” (not within a well-defined information architecture). This will apply out-of-the-box and custom classifiers to intelligently apply retention across your tenant by classifying content based on meaning and context. I haven’t seen this in any tenant I’ve worked with yet, but this will add another level of automation many organizations are waiting for! Once I see it, I’ll blog about it…
Licensing… the capability to auto-apply labels described in this post requires an Office 365 Enterprise E5 license (or an Advanced Compliance add-on) for each user who has permissions to edit content that’s been automatically labeled in a site. Users who simply have read-only access do not require a license.
If you don’t have the license for the auto-apply functionality or you have advanced logic to determine if a retention label should be applied that can’t be done in the auto-apply condition, refer to another post of mine where I show an alternative way to automatically set a Retention Label using Microsoft Flow: Setting a Retention Label in SharePoint from Microsoft Flow. This provides a lot of flexibility to include any kind of logic you may require to set the label.
Retention labels can be auto-applied based on 3 conditions:
- sensitive information types (both out-of-the-box and custom)
- keywords
- content types and metadata
This post describes the third option above to demonstrate the auto-apply behavior across several column data types and content types in SharePoint. Due to the fact that the retention label isn’t applied immediately (controlled by a back-end process that runs ~once per week), this is not a quick test to do. I’ve spent the time testing this, so I’m sharing the results and learning with you!
Please refer to the ‘Important things to know’ section at the end of this post for some key takeaways on this functionality. I will update the takeaways as I learn more.
Apply a Retention Label based on a Content Type
Content type called Contract document has been added to a document library. Retention label called Contract has been created and auto-applied based on the condition below:
ContentType:’Contract document’
The result? Within a week, any SharePoint site the retention label is published to had the Contract retention label applied to all documents with the content type of Contract document.
Apply a Retention Label based on a Choice Metadata column
A choice metadata column, ContractType, has been added to a library. I want to use one of the choice values to set a retention label. The auto-generated managed property from the search schema cannot be used in the auto-apply condition. You must manually map the crawled property to a RefinableString property (it’s queryable). For this example, I’ve mapped the crawled property generated for metadata column, ContractType, to RefinableString00.
Retention label called Hardware has been created and auto-applied based on the condition below:
RefinableString00:Hardware
The result? Within a week, any SharePoint site the retention label is published to had the Hardware retention label applied to all documents with a choice value of ‘Hardware’ on the ContractType metadata column.
Apply a Retention Label based on a Managed Metadata column
A managed metadata column, Department, has been added to a library. I want to use one of the term values, Senior Leadership, to set a retention label. The auto-generated managed property from the search schema cannot be used in the auto-apply condition. You must manually map the crawled property to a RefinableString property (it’s queryable). For this example, I’ve mapped the crawled property generated for metadata column, Department, to RefinableString99.
Retention label called Executive has been created and auto-applied based on the condition below:
“RefinableString99:’Senior Leadership'”
The result? Within a week, any SharePoint site the retention label is published to had the Executive retention label applied to all documents with a term value of ‘Senior Leadership’ on the Department metadata column.
NOTE: thru testing, a managed metadata column will only work as an auto-apply condition if it has been published from the Content Type Hub.
Apply a Retention Label on a compound condition
What about combining conditions? You can do this too! This test combined a content type name of Contract document with a choice value of Software. A retention label called Software has been created and auto-applied based on the condition below:
ContentType:’Contract document’ AND RefinableString00:Software
The result? Within a week, any SharePoint site the retention label is published to had the Software retention label applied to all documents with a content type of Contract document and a choice value of ‘Software’ on the ContractType metadata column.
Apply a Retention Label on a Date column <= Today
Can you include date logic in the condition? Yes. I have added an optional date column, DateExpired, to a library and want to apply the retention label once a date has been entered and is today or in the past. To filter on a date column, you must map it’s crawled property to a RefinableDate column (it’s queryable), so in this case I mapped it to RefinableDate01. A retention label called Expired Contract has been created and auto-applied based on the condition below:
RefinableDate01<=TODAY
The result? Within a week, any SharePoint site the retention label is published to had the Expired Contract retention label applied to all documents when a date either equal to today or in the past has been entered into the DateExpired metadata column.
Note: if a date isn’t entered in the column OR a future date is entered in the column, a retention label is not applied.
Important things to know:
Through my testing, I learned a few things that are important to understand and communicate to the Information Management team:
- The back-end timer process to update the retention labels only runs weekly. If it is your expectation that the label will be updated soon after the metadata or content type is updated, this is incorrect. In tenants I’ve tested with, the process runs in the early hours of the morning (UTC-6) on Friday or Saturday mornings.
- If a retention label is already applied on a document, the auto-apply process will NOT override/replace the label even if an auto-apply condition is met. Example: if you set the column Contract Type to Software and this auto-applied a label to ‘Software’, and then you subsequently change the Contract Type to Hardware, the label will not change to ‘Hardware’ if you had an auto-apply condition set to that. The original label, Software, would remain.
- The columns filled in when the label is auto-applied are: Retention label, Retention label applied, and Label setting. The Label applied by is not filled in.
- You can still manually remove the retention label by editing the properties. If you do this, the next time the back-end process runs, it will re-assess the document based on the auto-apply conditions and, if met, re-apply the correct label.
- A simple way to test out your conditions before creating your label policies is to enter the query directly in the Microsoft Search box thru the SharePoint UI. It will return the same results.
Although I’ve seen other posts where an auto-apply condition was based on a managed metadata term value, I’ve been unable to get this to work. I have a ticket open with Microsoft to understand the issue. Once resolved, I’ll update this post. I’ve updated the post with a managed metadata column.
Thanks for reading.
-JCK
Principal author: Joanne Klein, MVP
LinkedIn: https://www.linkedin.com/in/joannecklein/
Blog: https://joannecklein.com
Credit:Image by Jess Watters from Pixabay
Thanks so much for the time you’ve put in to this post! Just a quick question – your point 2 above is quite worrying. Would there be a way around this, for example when a metadata entry is changed this triggers a Flow that removes the retention label? So then your point 4 could happen and the new label would be applied.
I’m also interested in point 2, there’s a point here within Microsoft’s documentation https://docs.microsoft.com/en-us/office365/securitycompliance/labels that states:
“If there are multiple rules that assign an auto-apply label and content meets the conditions of multiple rules, the retention label for the oldest rule is assigned.”
I interpret this as how an auto-applied retention label is applied to a new un-labelled document.
If this is by design and intended it might by because it prevents manipulation of the retention period of the original record, although this would negate the ability to alert the retention period through different stages of its lifecyle.
If it is by design I feel that Microsoft should include another point in their documentation that should state something like this:
“If content has had an auto-apply label assigned in the past and metadata values are changed to meet different conditions a new auto-apply label is not auto-applied.”
I’ll see if i can raise that as a request in the Microsoft documentation and have them at least clarify. If i have any success i’ll report back via this post.
Thanks for another valuable post! I was triggered by the statement at the beginning of the blog “Microsoft has also announced machine learning classifiers…”. I actually noticed an Ignite session (BRKBRK3223) from 2018 where Microsoft outlined this capability (https://myignite.techcommunity.microsoft.com/sessions/65669). With such an advanced feature, I’m thinking why I missed it before. I am super excited to go hands-on when it becomes available.
Once again thank you for documenting all of this, it is invaluable!
I have one question on your use of the built in managed properties, have you tried creating your own queryable managed properties and mapping them to your crawled properties? Just wondered if that was an option as it’s possible some other process is already making use of the built in RefinableXXX managed properties and there’s (albeit a very small) chance a label could get applied with a “false positive”.
Hi Colin, no I haven’t tried that but I see no reason why that wouldn’t work. Thanks for the feedback!
-JCK
Hi Joanne. When you add your logic to the Retention Policy, do you add it without quotation marks (RefinableString00:Hardware), or with single marks (RefinableString00:’Hardware’), or with double quotation marks (RefinableString00:”Hardware”). I’m trying your process listed above using a managed metadata column and a choice column but neither are working for me.
Kieran
Check out point 6 under my “important things to know” section – I haven’t been able to get managed metadata to work. Choice will work, no quotes are required around any of it unless your value has a space in it… in that case put single quotes around the value.
-JCK
Managed Metadata works =) You can use the Term GUID or use RefinableString https://bobbyschang.com/2019/05/22/auto-classification-office365-label-metadata/
Awesome! Thanks Bobby!! I thought I tried the refinablestring, but couldn’t get it to work. Thanks for commenting – I’ll try again! (and wait a week) 🙂
Hi Joanne, I would really like to know your opinion on this scenario. If a label is auto applied under the scenario “RefinableDate01>TODAY”, then i want to auto-apply another label when this condition fails and refinabledate01 becomes equal to TODAY. Will it remove the old auto applied label and apply the new one which is matching criteria.
Hi Summaiya,
Once a label is auto-applied, it won’t be overwritten even if metadata changes to satisfy another label’s condition.
-JCK
Thanks Joanne for a quick response. Can we achieve the same by creating a retention policy instead of label? will retention label will win over retention label after refinabledate01 passed TODAY?
Summaiya
Depends. You need to understand the principles of retention to know what takes precedence. Reference: https://docs.microsoft.com/en-us/microsoft-365/compliance/retention-policies#the-principles-of-retention-or-what-takes-precedence
As always creat overview Joanne. Curious if you’ve ever had to apply different retention polices to SharePoint Online by geo? NAM want x, EUR want y, and APC wants z. 🙂
Hi TG,
No, I haven’t. Interesting scenario… the only way I can think of to do it is via PowerShell where depending on some property of your site indicating the geo, you would include it as a publish location in the geo-specific policy. Never tried it though. Would be interested to know how you solve this one.
-JCK
Hi Joanne-
First thank you for this post its been so helpful. My question is around working with the auto labelled content. I have setup auto labeling based on content type. The labels have been applied and I still have been able to work with the content and I don’t have an E5. I also have a test account with an A1 for faculty and it is able to work with the content that has been auto labeled. Where is or how would I know that I can’t work with the auto labeled content?
Thank you!
Hi Matthew,
As I understand it, it won’t *prevent* you from setting it up, however Microsoft will know that service is being utilized in your tenant and at license renewal time, you may be asked to “true up” for using it. However, I am NOT a license expert and that is best confirmed with your own Technical Account manager.
-JCK
Hi Joanne,
I’m just wondering, if you set up a label with auto-apply rules on it, can an end user still view and apply the label themselves manually?
Hello. Yes they can see the retention label in the UI and manually apply/change the label as well.
-JCK
Great, thanks! I thought that was the case, but a Microsoft team I’m working with are saying otherwise in a bid to get us to go with an approach that is easier for them to implement but doesn’t meet our business or statutory requirements.
I would be interested to know why they are saying this. I might not know all the details.
Hi Joanne, great article, i have a question regarding “automating conditions for auto apply”, i understand that you can use powershell to push the labels but how can i automate the conditional part ? let us say i want to apply label x, when content-type=B, i have about 300 labels, i will have to add these conditions manually unless there is a better way to automate this.
Hi Ganesh,
The ‘auto-apply’ capability happens when you publish the labels. Perhaps you could use PowerShell cmdlet: Set-RetentionComplianceRule (https://docs.microsoft.com/en-us/powershell/module/exchange/set-retentioncompliancerule?view=exchange-ps) I will likely blog about this as well.
-JCK
Hi Ganesh – should have said New-RetentionComplianceRule that you would use to add the rule to a retention policy.
-JCK
Hi Joanne, a auto retention label for the standard folder path of the teams chat files is working great. But there are only the document files labeled and the picture items are not included. So why does the query Path:”*my.sharepoint.com/personal*” AND Path:”*Documents/Microsoft Teams Chat Files*” not selecting all filetypes?
Hi Till,
Don’t know and hard to troubleshoot over this method. As far as I know, it should work but I’ve never tested it. I can certainly manually a retention label to “picture items” so there’s likely something else going on there. Good luck.
-JCK
Hi Joanne, What would be the best way to retain notes – when you have a retention policy that deletes items after 4 years. We want to be able to keep that content in outlook and not have it automatically delete
Hi Rich, I need to clarify the ask… what do you mean by ‘notes’. Where are the ‘notes’ originally stored?
-Joanne
Hi Joanne, I am referring to the notes (in the notes tab) inside Outlook
Hi Rich,
Full transparency, I was unaware of these notes – and I’ve never had a customer use them. That said, if they fall under the retention policy of the mailbox, then you won’t be able to get around their deletion unless you download a copy and store it elsewhere. Typically, a retention label applied to an email with a longer retention period would have the ability to override the mailbox retention policy if you needed to keep something longer (or indefinite), however I don’t know if you can apply a retention label to a “note” in Outlook – I’m suspecting not. Are you (your customer) insistent on using notes… seems to me there are other alternatives in M365 for sharing persistent notes that may suit better that could be managed outside of the retention policy. Without knowing more about the scenario, my guidance can’t be more specific.
-Joanne
Hi Joanne
Is there a way of automatically setting a default retention label to the document library in all OneDrive locations in a tenant or adaptive scope?
Hi Kalpesh. Yes. You could use a static scope to accomplish this for all OneDrive locations. In order to apply a retention label to all documents, you would need to auto-apply the label based on a condition that would always be met for library content… for example contenttype:document (or whatever else you wanted to use) and then publish it to all OneDrive sites.
-JCK
Alternatively, once Adaptive scopes is released, you could use the auto-apply label policy scoped to your adaptive scope.
-JCK
Thanks Joanne. Does contenttype:document apply to all file types including non Office 365 files like videos, csv & audio?
Also, does it apply to files in subfolders?
Finally, is there a way for users to bulk apply a retention label to multiple files in a OneDrive folder? If I select two or more files, the details pane (I) doesn’t open.
Hi Kalpesh,
A safer bet is to use the start of the ContentTypeId to make sure you catch all of the types of content that inherit from the base content type so… ContentTypeId:0x0101*
Yes, it will also apply to files within folders/subfolders.
Never tried bulk-updating files in OneDrive thru the detail pane. The fly-out pane isn’t opening for me either.
-JCK
Thank you so much, Joanne.