I am a Purview and SharePoint MVP and so when autofill columns were introduced into SharePoint, my curiosity was piqued to understand how Purview and SharePoint could jointly benefit from this feature.
From the Microsoft Learn page: “By using large language models (LLMs) through generative AI, autofill columns can save metadata automatically, streamlining the process of managing files and their associated information.”
“This metadata will improve library-scoped chat experiences and enable rule-based automation flows.”
I wanted to understand how/if Purview would be able to leverage the values stored in autofill columns to automatically apply a retention label based on those values.
Short answer: yes, but with significant caveats.
Realistic answer: although you can certainly auto-apply a retention label to files across SharePoint that have values in an auto-fill column (shown in this post), you must know the internal name of the autofill column and the column data type used in order for it to be mapped to a refinable managed property in the tenant-level search schema – a prerequisite for Purview. This is where some challenges enter the picture.
In this post…
- 1️⃣The Consistency Challenge
- 2️⃣The Scalability Challenge
- 3️⃣Walk-through of Autofill columns applying a Retention Label
- 4️⃣What about the ‘Organize this library’ feature of the SharePoint Knowledge agent?
- 5️⃣Parting thoughts
1️⃣The Consistency Challenge
There are many ways to automatically apply a retention label to content in SharePoint; however, if you want to use a piece of metadata to apply it, there is some back-end setup that must occur in the tenant-level search schema before Purview can use it.
Tenant-level search schema: https://<domain>-admin.sharepoint.com/_layouts/15/searchadmin/ta_listcrawledproperties.aspx?level=tenant
Consider the scenario where there are 2 libraries in the same site and the autofill feature has been used to create columns in both libraries. Library 1 has an autofill column called Document category. Library 2 has an autofill column called Doc category.
The internal name of these 2 columns in SharePoint is different even if the display names were changed to be the same after they were created.
I was pleased to see that when autofill columns are created from the Create column document library option, all spaces that are in the column name are removed before setting the internal column name.
From the example above, the 2 autofill columns have these internal names:
- Documentcategory
- Doccategory
This is where things start to go sideways… even if you change the display name of one of these columns to match the other after it’s created, it’s too late! The crawled properties created in the search schema are based on the internal name, not the display name. Because you cannot change the internal name of a column after it’s created, there will be different-named crawled properties created providing no consistency for Purview to work with.
In addition, the crawled property name generated differs depending on the column type chosen by the business user when creating the autofill column.
Example: Below are the 2 crawled properties automatically generated in the search schema that Purview should use (there are others generated) if the Document category and Doc category columns were added as managed metadata autofill columns:
- ows_taxId_Documentcategory
- ows_taxId_Doccategory
Conversely, these are the 2 crawled properties automatically generated in the search schema that Purview should use (there are others generated) if the Document category and Doc category columns were added as single line of text autofill columns:
- ows_Documentcategory
- ows_Doccategory
Since Purview LOVES consistency, this introduces a challenging problem.
The Purview administrator would have to know to map each of the above crawled properties into the same Refinable managed property before Purview could use it in a KeyQL condition to auto-apply a retention label on matching content. If a third autofill column was added in the future called Document ctgy (how will you know a business user has added it and what column data type they chose?), then the Refinable managed property previously mapped in the search schema would need to be updated to also include the new crawled property for the new autofill column (ows_taxId_Documentctgy or ows_Documentctgy depending on the column type).
The operational implications to Purview are significant which leads to challenge #2…
2️⃣The Scalability Challenge
You will probably want to apply a retention label on matching metadata to more than just 1 library or site. Typically, you would want to do it across all sites in your tenant or a subset of your sites by scoping the retention auto-apply label policy to the appropriate group of sites.
The power of Purview is its ability to find matching content across all scoped sites ensuring that regardless of where matching content is found across those sites, the right retention label will be applied.
Due to the flexibility of autofill columns for business users which is certainly a self-serve benefit (users can control the names, call the same type of content different things in different libraries, can pick the column data type), this flexibility has significant downstream impacts on reusability and, what this post is referring to, the ability to auto-apply Purview retention labels.
There is no guarantee that 2 autofill columns created in 2 different libraries (same site or different site) will have the same internal name or the same column data type. Due to this, Purview cannot easily scale across many libraries and sites – something it needs to be able to do.
Purview needs consistency to scale.
3️⃣ Walk-through of Autofill columns applying a Retention Label
To demonstrate, I’ll walk thru an example of applying a retention label to everything with a document category of Technology across any SharePoint site in scope. I’m using this example because I believe a document category will be a common autofill column. (something I’ve seen already with customers)
Whether the autofill column data types are Single line of text, Choice, or Managed Metadata, the issue still exists. Without controlling the internal display name of the column and knowing what the column type is, you simply don’t know in advance what the crawled property will be.
In the real world, I have NEVER used a Single line of text column to auto-apply a retention label. Unfortunately, I suspect that will be the easiest option for business users to pick when creating a column and using autofill. That column type allows anything to be entered into it (up to the character limit) making it a very bad choice for knowing what values will be there for application of a retention label. The autofill feature makes it slightly better; however, there are no guarantees what the values will be.
This is why managed metadata is a better option since there is some tenant-level control on the values that will be autofilled. This still doesn’t solve the problem.
Step 1: Autofill columns created for Document category and Doc category both as Single line of text
 |
 |
Step 2: The Crawled properties created in the search schema
Step 3: Map a RefinableString managed property to the crawled properties that Purview uses for applying a retention label
Warning: if additional autofill columns are added in the future with different internal names or column data types, the crawled property generated from the autofill column would have to be mapped to the RefinableString46 managed property above.
Step 4: Use the RefinableString managed property mapped above to apply a retention label, DEMO – Tech Guidelines, in an auto-apply retention label policy (Purview’s Records Management solution)
Step 5: Retention label is applied to content based on autofill value in the Document category column. Same label would be applied to other library in the site with the autofill value in the Doc category column.
4️⃣What about the ‘Organize this library’ feature of the SharePoint Knowledge agent?
The recently announced SharePoint Knowledge agent in Public Preview (link below) has an “Organize this library” feature that automatically suggests 3 autofill columns be added to a library based on its knowledge of the content in the first 20 files of the library.
This is certainly a good feature for enhancing content management within SharePoint; however, since the columns generated are all autofill columns, the same challenges I describe above remain.
Note: the “Organize this library” feature of the SharePoint Knowledge Agent currently does not remove spaces in the autofill column names it creates. Hopefully, when this feature goes to General Availability, the spaces will be removed in the internal column name. Only if you manually create the autofill column directly from the SharePoint library’s Add column feature are the spaces automatically removed (what I demoed above).
By removing the spaces automatically, it will create the internal column name as this: Document_X0020_category instead of this: Documentcategory
Link: Get started with Knowledge Agent (preview) – SharePoint in Microsoft 365 | Microsoft Learn
5️⃣Parting thoughts
This is not a new problem. To be clear, lack of metadata and metadata consistency across a tenant has been an issue long before autofill columns were introduced. Thanks to the autofill feature, the business-user burden of setting metadata has been significantly reduced and that alone brings significant content management and AI benefits.
However… although having metadata is better than not having metadata, the issue remains that unless the metadata structure is consistent, downstream features and solutions needing to scale will struggle to take advantage of it.
Purview retention label application is certainly one of those downstream solutions.
Until autofill columns can leverage site content types/site columns that have been curated and controlled by the SharePoint information architects (consistent names, consistent, pre-planned data types), the Purview auto-apply retention label story remains challenging. We’re early on the roadmap for autofill columns so it is my hope that over time, this will happen.
Until then, please be aware of this limitation if it is your intent to be able to leverage some of the autofill metadata for auto-application of a retention label.
Thanks for reading.
-JCK