Retention in O365. The new way.

Preservation Policies in O365 (3)Blog post: 3 minute read.

Retention. It’s not the most exciting topic in the world of Office 365, but it is a very important one. It’s the feature that ensures your organization is keeping things for as long as you should and, just as important, disposing of things as soon as you should. My current focus within Office 365 is to understand the new Retention features and how they work across the O365 collaboration services.

Retention OptionRetention policies are administered from the Security and Compliance Center in O365 within the Data governance section. Whether the policy has been defined via a published label (Label Retention across O365) or by creating one directly within this Retention section, they will all appear on this page. A Retention policy can include content from not only Exchange mailboxes, public folders, Skype conversations, SharePoint sites and OneDrive for Business content but also Office 365 Group mail and files. A retention policy is the only feature that can both retain and delete content across Office 365. As content is spread across these services, this is your “one-stop shop” for retention. A very good thing.

Here is the official Microsoft link describing what a retention policy is and when you might want to have one: Overview of retention policies.

Follow along for a quick walk-thru of adding a retention policy for an O365 Group’s site and the ‘magic’ behind the scenes.


#1 – Setting it up

Here are the steps to create a location-based retention policy:

  1. Navigate to the Security and Compliance Center. (https://protection.office.com)
  2. Under Data Governance … Retention … click Create(+).
  3. Follow a 4 step wizard:
    1. Enter a policy name and description.
    2. Decide if you want to retain content for a period of time and/or delete it after a certain length of time. Content can be either retained forever or for a specified period of time based on when the content was either created or last modified. You can specify advanced retention options whereby retention is applied to content containing certain keywords or phrases or sensitive information. There will be more capabilities added for this in the future with searchable managed properties.
    3. Choose whether you want to preserve content from all locations (Exchange email and public folders, Skype for Business, O365 Groups, SharePoint and OneDrive for Business document libraries) or select locations.   Note: you provide the Site Collection URL and not a web URL when specifying a SharePoint site.
    4. Optionally turn on preservation lock. This will make the policy locked preventing it from being turned off. This is what give O365 the SEC 1784 compliance certification!

A message will display stating that it can take up to 1 day for the retention policy to be applied to the locations you specified. In my experience so far it is applied much sooner than that. However, until it is published it will have a status of On (Pending).


#2 – Preservation Hold Library

Important thing to know about a retention policy is regardless of how it was applied (by either classifying data with labels or by applying a policy to a location as was done in step 1 above) the end-user working with the content really has no idea the policy is in effect from a content editing perspective. They can continue to addchange content as they normally would, all the while the retention policy is working silently in the background ensuring the content is being preserved in another hidden location.

This hidden location? The Preservation Hold library.

When I edit the document in either of the above sites, it will allow me to edit the document, but adds an item into a special library created on the site called the Preservation Hold Library (/sites/yoursitename/PreservationHoldLibrary). It creates the library in the site (or subsite) you are making the change in and it only creates the library when it is required (I.e. the first time a change is made and an item needs to be inserted into the library, it will create the library if it doesn’t already exist). You must be a site collection administrator to see this library.

Here is the metadata on a Preservation Hold Library (I’ve highlighted the ones that are key to the preservation policy):

preservationholdlibrarycolumns


#3 – Policy is deployed… now what?

New Content

Any content added to the site after the retention policy was put into effect will be preserved after deletion. Changes on new content aren’t copied to the Preservation Hold library the first time it’s edited, only when it’s deleted. (Unless you have versioning turned on)

For example, on my O365 group I uploaded a document (my resume in the screenshot below) after the policy was deployed. I made 2 separate edits (saved each time) and then deleted the document. Only after I deleted the document did these 3 items get added to the Preservation Hold library on the site.

NetNewDocumentDeletyed


Existing Content

If an item exists at the time the policy was put into effect, the first time you make a change to the document it will insert an item into the Preservation Hold Library list (what the item was before the change). Subsequent edits on the document will not insert an additional item into the list, however if the document is ever deleted, all versions of the document will be inserted into the list as separate items.

For example, if this is the version history on a document at the time of deletion:

FileVersionHistory

… when the document is first edited, a snapshot of the document and its metadata as it existed prior to the change is inserted as an item in the Preservation Hold Library list (the last item identified as 40 minutes ago in the diagram below). Once the document is deleted, all previous versions are inserted as items into the Preservation Hold Library:

PreservationHoldLibraryAfterDelete


#4 – Removing a Preservation Hold

Content in the Preservation Hold Library list will be removed when one of these things happen:

  1. A policy administrator has changed the rules for what’s covered by the policy and the content no longer complies.
  2. The policy has been disabled.
  3. The policy retention end date has been reached.
  4. If retention is based on a label, the label was removed/changed.

The retained content is not deleted immediately – this is done by a timer process.


#5 – Legacy features

Things are changing. What we used to use for Retention and Deletion is no longer recommended. Instead, Microsoft is recommending you create a Retention policy for these things: (excerpt from the Microsoft link above)

Exchange Online:

SharePoint Online and OneDrive for Business:

Also, if you have any legacy preservation policies defined, they will still preserve the content in the Preservation Hold Library as described in a recent Office support article:

What happened to preservation policies?

If you were using a preservation policy, that policy has been automatically converted to a retention policy that uses only the retain action – the policy won’t delete content. The preservation policy will continue to work and preserve your content without requiring any changes from you. You can find these policies on the Retention page in the Security & Compliance Center. You can edit a preservation policy to change the retention period, but you can’t make other changes, such as adding or removing locations.


SUMMARY

It is clear Microsoft is serious about Data Governance across the entire suite of O365 services. Information Management teams are looking to us, as O365 Consultants, to make sense of this ever-changing landscape for them. I’ll continue to watch this space closely as I suspect more capabilities and features will be introduced over time.

Thanks for reading.

-JCK

Advertisements

11 thoughts on “Retention in O365. The new way.

  1. Hi Joanne – good article, and timely for me as I’ve been working with Retention this week.

    I wonder have you encountered the following. If a Policy includes all SharePoint sites the Exclude site option does not seem to work. This obviously causes issues if you subsequently want to remove a site – maybe to place it elsewhere due to company reorg – the site cannot be deleted, even if you placed no content there. Have a ticket open with Microsoft on this

    1. Hi Ian. Thank you. I haven’t tried this. When I get a few minutes I will give it a try and let you know. So can I assume the policy prevents you from moving it to new location? Prevents you from renaming it?

  2. Great piece, very practical. We’re compiling our model for content types and retention for a migration to O365. Your article makes a number of things clear. I’m just starting to look at the use of Keywords (including content types) and sensitive information types. Do you have much experience with these yet?

    1. Hi Russ, To my knowledge, you cannot write a keyword query against the Content Type managed property from the Search Schema in SharePoint. (This is ideally what we want and something that the product team is currently working toward). Due to this, I believe the retention options are not at a point to be a 1:1 replacement for the traditional Information Management Policies we’re used to using .. It will partially depend on what your retention requirements are – if they are based on content type, I believe you are safe still doing the traditional IMP technique and I’ve been told there will be a transition piece put in place to transition from them to the new retention label model in O365. Hope this helps.

  3. It does help, though with a bit of a sigh aimed at Microsoft… The page on labels specifies “The conditions available now support applying a label to content that contains specific words or phrases,” but I had hoped that it would extend to other kinds of Keyword queries. Both the RM model and user experienced are helped if such things work as much as possible via a single technology. So I’m glad that it is on the table.

  4. A very helpful article. Thank you. A question if I may: I don’t want any content from our O365 site deleted automatically at any time so, when I set up my Retention Policy, I say ‘No’ to ‘Do you want us to delete it after this time.’ That much seems clear. However, what I’m not sure of is, if I say ‘No’ here, what happens to items that have been moved to the Preservation Hold Library or Recoverable Items Folder at the end of the Retention Period? Are they permanently deleted or not?

    1. Hi David,
      I apologize for this tardy response. If you don’t want the content deleted at any time, you will need to select ‘Forever’ as the retention period rather than a number of days/weeks/months/years. If you do this, you won’t get the option to ‘Do you want us to delete it after this time?’

      Hope I understood your question correctly and that it helps!
      Joanne K

      1. Hi Joanne,

        Thanks for your response in clarifying the nature of retention policy. I’ve read this article (http://blog.enowsoftware.com/solutions-engine/preservation-policies-in-office-365) on preservation lock, as well as other Microsoft blogs around preservation/immutability of data. It all seems to point to immutability and not permanency of data.

        None of the articles that I can find explains where “legal hold” data is retained if no license is assigned to a user. I’m sure someone has to pay for the costs of storage at the least, but I can’t get clarity on 1. where this preserved user data is stored and 2. how the usage is billed if you want to keep data forever.

        I recently received a response from MS partner community that “If we want to keep data forever in O365, we should keep the O365 subscription is active” – which eludes to an assignment of license to prevent data expiry – and that retained data is kept within the users allocated storage.

        If you have any thoughts on this, I would appreciate it.

        Best Regards,
        Navs

  5. Hi Joanne – thanks for clearing up some important misconceptions. Would you be able to guide me on how preservation/retention policies protect onedrive data after a user is deleted (and passes the 30 day data expiry)? I cannot seem to get a clear answer on this and my assumption is that the onedrive data is gone (site cleanup) after a user is deleted (after 30 days). Can you please clarify this if possible?

    1. Hi Navs,
      To my knowledge, a retention policy will not protect a User’s OneDrive for Business data past the 30 days.
      There are two possible scenarios: the user account still exists, or it will be deleted…

      If the user account still exists, you can use a retention policy. If the latter, OneDrive for Business content will be deleted 30 days after the account is deleted like you said. You will need to migrate the OneDrive content to a new location in order to retain it.

      Another possibility is to apply a legal hold to content. Then it cannot be deleted by a user and will not be deleted even if the OneDrive account is deleted.

      Hope this helps.
      Joanne K

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s