We help Microsoft-centric enterprises fully adopt the cloud & adapt to new ways of working.
\

CATEGORY

Migration

Is drive shipping best for large PST migrations to Microsoft 365?

Posted on by Alethia Erchen

Do you need to migrate thousands of PST files, really big mailboxes or large email archives to Microsoft 365 (and is drive shipping the answer?)

What is a PST file?

A PST file is an Outlook Data File (.pst) that can be used to store messages and other Outlook items.  From the ‘outside’ they look like a single file, but inside they have the structure of a pseudo mailbox.

Larger organisations in particular can have a legacy of many thousands of PST files sitting on individuals’ local drives, home drives and file servers.  They would historically get created as a way of staying within mailbox quota (when Exchange mailbox quotas on premises where perhaps not as generous as they are online).  PST files were also used as a convenient way of preserving mailbox contents – perhaps after an individual had left the company.   In fact PSTs used in these scenarios are widely considered a pest – especially when organisations switch to the cloud.

PST files can also be used as an interim store when migrating between platforms, as many mail systems can export emails and attachments into PST format, and conversely, import content from a PST file.  It is this ‘use-case’ that is the subject of this particular article…..

What is drive shipping?

Microsoft’s drive shipping service is multi-step process via which large amounts of data can be uploaded into Microsoft cloud via an interim device which is physically shipped to a Microsoft location.

This is how it works:

Instead of transferring data across the network, the technique involves writing PST files to a hard drive along with a mapping file (for example, migrate PST <filename> to <username> primary or archive mailbox).

The (encrypted) hard drive is then physically shipped to a designated Microsoft location from where data centre personnel pre-stage the contents into Azure.  The files are then ingested into Exchange Online according to the supplied mappings.

If you have TBs of emails you want to move into Microsoft 365, drive shipping via PSTs is an option open to you.

This Microsoft article goes into all the different iterations of what you need to do and the cost of using this service.

The same technique can be used to perform migration of mailboxes that are over 100GB in size or if the user’s mailbox contains one or more messages that exceed the 150-megabyte (MB) message limit (in which case resorting to PST files is recommended).

Using interim PSTs is also an option for migrating the contents from large email archives such as Enterprise Vault and EMC SourceOne, or email journals form platforms such as Mimecast.

The question is:  Is drive shipping PSTs a good option for your large email archive migration? 

Here are the pros and cons:

Pros

  • It’s low cost…on paper*. The cost to import PST files to Microsoft 365 mailboxes using drive shipping is $2 USD per GB of data.
  • It minimises impact on your network: if you have a sub-optimal network that cannot handle large amounts of data being transferred, using a data drive to ship PSTs to Microsoft negates any network concern. Even on networks that can support around 500Mbs you can experience slow performance when you start to drive a large migrations alongside regular user activity.
  • It avoids the impact of Microsoft throttling:Microsoft applies throttling to avoid overloading its servers. Although you won’t experience the effect of throttling when using native Microsoft mailbox moves to migrate your mailboxes, many email archive migration solutions use the EWS protocol to move your data, and this protocol is subject to throttling, although Microsoft has made throttling easy to ease off during the course of a bulk migration.

Cons

  • *It can work out expensive:At face value, $2 USD per GB is cost-effective, for example, a 20TB project would be $40,960 to ‘drive ship’, but this does not include the added overheads of getting your data onto the drives (see next point).
  • PST preparation is labour-intensive. Suffice to say that manually extracting data from archives into PST files and then preparing them for upload can be super time-consuming.  Native tools for extraction out of third-party archives (such as the Enterprise Vault extraction wizard) are slow and not geared up for performing automated mass exits. Once you’ve extracted files, you’ll need to make sure they are prepared properly for Microsoft.  This includes the creation of a mapping file, so Microsoft knows what files(s) belong to who, and where you want them putting.  Check out the steps you’ll need to carry out.  Whilst it’s possible to automate the PST extraction and preparation process using third-party migration software, you’ll need to factor this additional cost in.
  • It can take a long time:You’ll need to allow 7-10 days for your data to be uploaded from the drives into Azure (as we said earlier, this is where your data is pre-staged) and then Microsoft offers an ingestion rate of 24GB per day.  Using our 20TB example, this means your PSTs would take 860 total days to ingest.
  • It introduces an element of risk:When using multiple hops and manual interventions to move your data, there’s the potential for things to go wrong.  Even though drive shipping uses Bitlocker encryption to protect your data in transit, there are many other steps that introduce the potential for human error, this includes the process of babysitting the extraction into PST files from your archive and the mapping of PST files to their owners.  This, combined with the fact that extraction tools typically have no inbuilt error-checking, are unable to recover in the event of a failure, and no auditing, will make it difficult for you to prove chain-of-custody.  Oh, and did I mention that PSTs as an interim file construct are prone to corruption?
  • Your source data needs to be static. If you’re migrating the contents of an email archive using drive shipping via PSTs you’ll ideally need to make your archive static during the course of the migration.  This means stopping any archiving activity for the duration of your archive project, otherwise you’ll have the overhead of subsequently migrating any additions to your archive.  We’ve encountered several projects where stopping archiving is not possible.
  • Shortcuts aren’t being addressed (and create confusion). You will need to have a game-plan for dealing with the shortcuts (also known as stubs) that typically link to archived items.  Many enterprises end up migrating shortcuts along with regular emails into Exchange online mailboxes.  Whilst in most cases it’s possible to retrieve the full item across the network from an on-premises archive whilst your migration is taking place, you’ll have various issues that emerge once your PSTs have been uploaded into Microsoft 365.  This includes broken shortcuts (assuming at some point you will decommission your on-premises archive) and legacy shortcuts that can appear along with the full migrated item in the event of any eDiscovery exercise.
  • Other limitations:
    • Message Size Limits of 150MB
    • No more than 300 nested folders
    • Doesn’t support Public Folders
    • You don’t get flexibility on where your data is migrated to destination and split of data
    • Volume restrictions of up to 10 TB
    • A maximum of 10 Hard drives for a single import job

So should we use drive shipping for our migration?

In summary, the only time we can see drive shipping using PSTs as being beneficial is if you have:

  1. Very slow network connectivity
  2. Lots of inactive data to migrate. For example, archives belonging to ‘leavers’

Our email archive migration service uses a series of techniques to mitigate the impact of Microsoft throttling, enabling us to move archives directly from your archive into Exchange Online (either primary mailboxes or archives) at a rate in excess of 3TB a day.  There’s also no overheads or time delays involved by extracting into PSTs first.

We can also schedule migration activity to coincide with less busy times on your network.

Also, the fact that we can move your data in one step, direct from source to target avoids the non-compliance risk of interim storage and human error.

We can also help you avoid moving everything.  For example, by applying date ranges.

You can also avoid creating a storage overhead in the cloud by managing where data gets migrated to.  I.e., by moving messages over a certain age into archive mailboxes or moving PSTs belonging to leavers into a separate (but indexed) Azure-based store.

On a final note, using interim PSTs is also an option when migrating journals from services such as Mimecast and Proofpoint, but there are a few things to watch out for when migrating into Microsoft 365.  You can find out more about migrating journals to Microsoft 365 in this article.

Find out more for your PST migration project

Get in touch with our migration experts for an unbiased chat on the options open to you.

Get in touch

Is there a shortcut to email archive migration?

Posted on by David Kellett

FACT: If you migrate legacy shortcuts into Exchange Online, it’s highly likely they’ll ‘still work’ if your archive remains on-premises

If you move shortcuts along with regular emails as you migrate mailboxes to Office 365, nine time out of ten they will still work: 

That is to say, when you double click on the shortcut, the original full email will be retrieved across the network from the on prem archive.

For example, with Enterprise Vault (EV), a user can still recall/retrieve old, archived email items using EV shortcuts. They can also use the EV Search service to access and search their archived emails as part of integrated search within the Outlook client or via a web URL in a browser.

The downside is that post-migration:

  • you can’t archive anything new in EV 
  • you can’t use web access to access the shortcuts (as Veritas doesn’t control the OWA servers)
  • end users cannot restore or delete archived email items from shortcuts

Other archive vendors may behave similarly.

There are a number of other downsides:

It’s unsupported

Microsoft won’t support this set up – why should they?  As indicated above – the archive vendor cannot control the Microsoft environment and will therefore be limited in what it can offer.

Over time, your on-prem archive will become unreliable and expensive

“Although shortcuts to archived emails would still work across the network after we migrated to Microsoft 365, the prospect of maintaining a dedicated email archive on-premises was a non-starter.  The SAN it used was becoming increasingly unreliable and the overheads of maintaining it were costly.”

According to Stephen Appleby of An Post, the state-owned provider of postal services in the Republic of Ireland, “Although shortcuts to archived emails would still work across the network after we migrated to Microsoft 365, the prospect of maintaining a dedicated email archive on-premises was a non-starter.  The SAN it used was becoming increasingly unreliable and the overheads of maintaining it were costly.”

Not surprisingly, COVID-19 has also highlighted the challenges of maintaining and backing up systems that are sitting in a server room in the office.

Annual software maintenance and support costs can also escalate, and high costs don’t necessarily mean you get a high-quality service in return. 

For example, EAS, one of the first email archiving solutions on the market, has effectively ‘changed hands’ five times (from Educom – Zantaz – Autonomy – HP – Capax ….. ) Other solutions have become similarly acquired and/or merged into an existing portfolio, with each change risking an uncertain roadmap and lack of focus on development and support from the new ‘owner’.

You’ll have a separate place in which you’ll need to manage retention and eDiscovery

On-premises archives (and to some extent, hosted archives) are of a ‘technical’ nature when it comes to implementing retention policies and performing eDiscovery. 

It’s likely the IT department will still need to manage retention policies and do eDiscovery. 

We frequently encounter IT teams that spend hours responding to eDiscovery requests.  For example, a client using EAS found its eDiscovery service was simply not up to the job for the volumes they had, with searches taking hours and hours to execute.  The suggestion on the part of the archive vendor was to beef up the IDOL search servers available – something that would have been prohibitively expensive.

Likewise, the job of implementing retention policies typically falls on the shoulders of the IT team, with little guidance on deletion policies from the legal department.

The great news is that Microsoft 365 offers a compliance and eDiscovery capability that is becoming super-user-friendly, to the point that you can now shift responsibility for compliance-related activities OUT of the IT department.

The other big benefit of moving the contents of legacy archives into Microsoft 365 is that you can perform eDiscovery in one location.  Having multiple different locations to search and then reconcile introduces both time and risk, which brings us onto the next issue:

Ultimately, is it worth having a separate archive?

Those in the business of providing dedicated email archive services would, of course, argue for the need to have a separate archive in the cloud.

For example, there are Veritas partners that offer services to host the storage component or your entire EV environment in Azure.

Also some vendors offer a cloud-based version of their previously on-premises archive.  For example, you can migrate EV into Veritas’ hosted service, Enterprise Vault.cloud.

There are also hosted archiving services from organisations like Mimecast and HubStor (now part of Veritas).

This is where the blurring of the boundaries between archiving and backup come into play and that is a separate subject again.

Another factor with each of these approaches is that they require some form of change on the end-user’s view – i.e. the concept and convenience of shortcuts goes away.

Why you should move your archives to Microsoft 365

Unlimited capacity

The storage capacity benefits of using a separate archive have fallen by the wayside with Microsoft 365.  There’s very few email users whose mailbox and historic archives can’t be accommodated by Microsoft 365 licencing:

  • A Primary Mailbox can grow up to 100GB (E3 and above)
  • An Archive mailbox in theory has no cap on it

No charge for leavers

If you have archived mailboxes belonging to leavers, you can take advantage of inactive mailboxes and place them litigation hold (or specifying an appropriate retention policy) to retain their emails indefinitely (or as required) without incurring a licence penalty.  

The average ration of ‘leavers’ to ‘current staff’ archive mailboxes in the typical archive can be as much as 4:1, so this amounts to a significant cost saving.

Advanced Compliance Capability

As we said earlier, the compliance and eDiscovery capability now available in Microsoft 365 is ‘up to the job’ of handling most information requests and data protection needs.  Importantly – Microsoft’s trajectory is to provide services that don’t need as much input from the IT department.

If that is enough to compel you to shut down your on-prem or hosted dedicated, third-party archive completely and, ideally, get the contents of your archive into Microsoft 365, this is what you need to know:

Successfully migrating email archives into Microsoft 365 requires care (there is no real shortcut)

Migrating the contents of a legacy archive into Microsoft 365 is a speciality for Essential and something we’ve been doing since 2005.

One thing you’ll discover is that there are a few different routes you can take.  

For example:

  • Do you migrate shortcuts into online mailboxes using mailbox moves, and then replace them with the corresponding archived item later?
  • Do you migrate archives first?
  • Where do you put emails that users have deleted from their archive (but are still in the archive)?
  • How do you treat mailboxes that are too big according to Microsoft’s licencing rules and throttling measures?  This can become an issue when you ‘re-hydrate’ a shortcut with a much bigger archived item.
  • What about shared mailboxes, public folders and journal archives?

The answer to each of these questions will vary depending on a lot of things, including the capabilities of the archive you’re moving from.

For example, HP ACA and AXS-One are archives that need extra care and steps to ensure the best compliance and end user outcomes

“Essential worked out the best strategy for migrating our archived emails into Exchange Online.  We received top notch documentation from Essential’s consultant, Toni Mallin, and this enabled us to complete our own migration with very little assistance.” 

According to An Post, whose legacy archive used HP’s ACA (Autonomy Consolidated Archive), “Essential worked out the best strategy for migrating our archived emails into Exchange Online.  We received top notch documentation from Essential’s consultant, Toni Mallin, and this enabled us to complete our own migration with very little assistance.” 

As with An Post, the cost of a migration is always a consideration and we are able to offer a service that enables your IT team to easily oversee elements of the migration and reduce costs.

One thing for sure is that the cost of migration will be significantly less than maintaining an on-prem archive and performing countless eDiscovery searches.

Find out more about our email archive migration services.

Request a personalised full product demo

Get in touch

Sticky Bit 4: Migrating On-Prem Public Folders to Microsoft 365

Posted on by Ross Moorhead

Essential has worked on some of the largest Public Folder migration projects in the world.  Here’s a few tips from our gurus:

A few years back you didn’t have an option to migrate your legacy public folders to Microsoft 365 – in fact public folders on-premises were to be end-of-lifed.  SharePoint was initially tabled as an alternative, but this didn’t ‘wash’ with a lot of Microsoft customers because it didn’t offer the same functionality and was over-complicated.

Microsoft quickly changed its position (no doubt following uproar from lots of disgruntled customers) and now you can take advantage of modern public folders – a service that seems to be hanging together reasonably well and growing bigger in capacity all the time.  It’s now 100TB in total – it started out at 2.5 TB and then 50TB so it’s always worth checking here Exchange Online limits – Service Descriptions | Microsoft Docs!

As you might imagine, there are some caveats, clean-ups and other considerations that come into play if you want to make the move.

But first off, it’s worth getting a bit of background on the modern public folder construct vs traditional public folders:

The Modern Public Folder service is very different from the Public Folder database architecture in an on-premises environment.  It basically uses regular mailboxes that are automatically linked together and load-balanced (for Microsoft 365) as your Public Folders grow in size.  Being regular mailboxes they also benefit from being part of data availability groups (DAGs) instead of having to undergo painful public folder replication.

Here’s how the modern public folder to Microsoft 365 architecture works:

  • You kick off with a single, Primary Public Folder (PF) mailbox (which can grow up to 100GB in size)
  • Microsoft 365 detects when a PF mailbox is approaching the 100GB limit and uses an auto-split feature that creates a linked Secondary PF ‘overspill’ mailbox.
  • As the next mailbox fills up, another PF mailbox is added and content is automatically re-balanced across all the mailboxes.
  • This expansion continues until you hit an overall limit (at the time of the last update to this article it is 1,000 public folder mailboxes and 100TB in a single Microsoft 365 tenancy).
    See this page for the latest info: https://technet.microsoft.com/library/exchange-online-limits(EXCHG.150).aspx
  • A PF hierarchy is maintained alongside the PF contents in the Primary mailbox.
  • This hierarchy is updated to reflect the new location of items as new PF mailboxes are added and as content gets ‘re-balanced’ across the available mailboxes.
  • Read-only copies of the PF hierarchy are also stored in each of the Secondary PF mailboxes and these are kept in sync with the Primary using Incremental Change Synchronisation (ICS).

The key thing to note that is that as far as users are concerned, although the Public Folder to Office 365 mail comprise multiple, ‘lashed together’ mailboxes, they can be viewed and navigated as a single, logical entity.

This is a really great PowerPoint by MVP Peter Schmidt that describes the whole thing in more detail:

https://www.slideshare.net/petsch/modern-public-folders

See also this Microsoft document for details: https://docs.microsoft.com/en-us/exchange/collaboration/public-folders/limits?view=exchserver-2019

Planning Your Migration

Can you migrate?

If you’ve already updated to using modern PFs on premises and are using Exchange Server 2013 CU15 or later, Microsoft has a public folder migration solution using batch migration scripts as described in this article:

https://docs.microsoft.com/en-us/exchange/collaboration/public-folders/migrate-to-exchange-online?view=exchserver-2019.

If you are using ‘old school’ PFs (aka legacy PFs) hosted on Exchange 2010 SP3 RU8, you can follow these instructions for batch migration.  

https://docs.microsoft.com/en-us/exchange/collaboration-exo/public-folders/batch-migration-of-legacy-public-folders?view=exchserver-2019

Whichever version you’re migrating from you’ll need to run quite a few separate scripts (including a final synchronisation and switch which requires downtime) which means it can be quite complicated to use.

 

Do an Inventory and Have a Clean Up

Some of our customers store vital customer records in PFs.  They also have a lot of rubbish in them and migration is a great opportunity to do a sort out.

Start by doing an inventory of your PFs at a ‘high-level’, and get statistics such as size, item count, owners, permissions and last accessed dates.

In order to make solid and defensible decisions around whether content can be deleted prior to migration you’ll need to do a LOT of deeper digging, however gathering initial meta-data can give you some excellent pointers.  For example:

  • Removing empty and duplicate folders can be a quick fix.
  • Orphaned folders with an old last accessed date are a very obvious candidates for a clean up.
  • Knowing the owner of a PF (assuming it’s not ‘Administrator’) can help signpost who you need to contact in order to see if content can be disposed of.

As ever with records disposition decisions, seek to get the relevant data custodians to call the shots – don’t go it alone!

Bear in mind that a potential downside to deleting or excluding older/stale contents from your migration is that you could create an eDiscovery headache later. For example, an HR dispute may refer back to employment terms and conditions, pension fund arrangements, etc, that were published decades ago.

Analyse Your PFs for Potential Glitches

Given the inherent differences between the architecture of old PFs and Modern PFs, you’ll need to spend some time eliminating things that will upset the migration process. For example:

  • Check for stale permissions
  • Check there are no orphaned PF mail objects or duplicate PF objects in Active Directory
  • Check PF names – syntax errors in your legacy PF naming convention can cause problems. For example:
    • If the name of a PF contains a backslash () it will end up in the parent PF when migration occurs.
    • Trailing whitespaces within Mail enabled PFs and commas in the Alias field will also create synchronisation problems.
  • Check all mail enabled folders to see that they have the right proxy address.
  • If you have any forms, these need to be exported and re-imported into Office 365
  • If users have PF ‘favourites’, they will need to document these before you cut over, as they will disappear

Chunk Up Your Legacy Folders to Slot Nicely into the New Separate Mailboxes Model

As we said earlier in this article, Office 365 performs an auto-split and load-balancing function as PFs approach 100GB in size, but this process can take up to two weeks to complete.  This is not usually a problem when you are populating a PF during ‘normal use’, but when you’re in a midst of a wholesale migration, you’ll be chucking data into Office 365 PFs at a rate of knots, and Office 365 can’t recalibrate itself fast enough.

Common to all migration approaches, therefore, is the need to take the Office 365 PF size restriction of 100GB per mailbox into consideration and effectively run scripts to ‘chunk up’ your PFs into separate PFs that are less than 100GB in size before you start your move.  We suggest you check that your ‘chunks’ are split according to logical subfolders.

Don’t overlook that fact that some of the items in PFs may be archived, as this will not only impact how you do your migration, it will also impact your sizing analysis (as shortcuts to archived items can be a fraction of the actual item size).   Check the message class to do this – e.g. IPM.NOTE.EnterpriseVault.Shortcut

There are many other considerations to take on board to ensure the best outcome post-move, such ensuring optimum retrieval times by putting PFs in a geographic location that’s near to users that will be accessing it.  Ensuring the number of people accessing PFs is kept below 2,000 per mailbox is also recommended.

Post-move you’ll need to do lots of checking and you might also need to re-mail-enable mail-enabled PFs post migration as this attribute might not get migrated.

You can find other considerations here:  https://technet.microsoft.com/en-us/library/dn957481(v=exchg.160).aspx

Find a third-party tool or a migration partner.

There’s a few tools around that can help you with your migration and the best ones start by enabling you to do an up-front analysis of the state of your PFs so you see (and address) what might ‘trip you up’.

The one we particularly like also has a neat algorithm that lets you map your current public folder hierarchy into new modern public folder arrangement.

Another area where you might need assistance is if you’ve been archiving public folders in the past, for example, using Enterprise Vault or EAS.   At a push you can export archived content to PST files and use these as a mechanism for uploading on-premises PFs into Microsoft 365, but you need to know what you’re doing when it comes to splitting your PFs into ‘mailbox chunks’.

Get in touch with our team for a chat to see if we can help with your public folder migration (or at least point you in the right direction).

Let us migrate your Public Folders to Microsoft 365 (or elsewhere!)

We can simplify your Public Folder & Public Folder Archive migrations – or help you migrate to alternative platforms like Azure  Get in touch to discuss your options.

Get in touch

Check your cloud storage vendor escape route

Posted on by Essential Marketing

Are you being locked in by your cloud vendor?

Whenever I get on a flight I always count the number of rows to the nearest exit, so I can grope my way out of a smoke-filled cabin if the worst should happen.  A totally pointless exercise, as in reality I’d be toast, but at least it makes me feel better.

What is worth doing is checking your exit route if you’re planning to store your content in a hosted cloud service.

A common tale of woe relates to hosted email journaling vendors, whose built-in export tools are simply not up to the job of wholesale extraction when the customer wants to ‘move on’.

“It took us between 16 hours to a day to extract just one mailbox into a PST, which then needed to be re-imported.”

“We had to run a series of searches using the “from address” to collect all the emails belonging to each user.”

By all accounts, data extraction is not a fun exercise when you’ve got TBs of data to move.

Check your exit route

What’s involved in getting your data back out of the cloud has to be a primary consideration if you are planning to migrate into it.

Ask your prospective cloud vendor these questions:

  1. How easy will it be to get my data out,
  2. How quickly can I get at it? Will it be over the network or on a disk?
  3. What about chain-of-custody during the extraction process?
  4. How will I know I’ve got everything back?
  5. What format will it be in when I get it back?
  6. How much will it cost?

Cloud storage vendor escape route

If you’re stuck in a hosted journal service, or are contemplating your best options for zero lock-in cloud storage, get in touch!.

Get in touch

5 Things to Consider When Doing an Office 365 Migration Involving Email Archives

Posted on by Essential Marketing

If migrating user mailboxes in a hybrid environment wasn’t challenging enough, migrating email archives to Office 365 can introduce all sorts of complexity you hadn’t bargained for.   

Here’s 5 things we’ve encountered when helping organisations with the end-to-end migration process.

1. Migrating Shortcuts Can Be a Useful Option

In most migrations to Office 365 there will be a period of time when co-existence with your on-premises archive is required.

Despite what some vendors may say, most on-premises archives can be configured to retrieve when the mailbox is moved to the cloud*.

Allowing co-existence will alleviate most of the time pressures in needing to migrate users’ archives at the same time as their Live mailbox. Large archives, in particular, may take more than a day to migrate and users do not want to lose access to their legacy data for any length of time.

Additionally, if you’re migrating user mailboxes into Office 365 using MRS or third-party tools, you’ll most probably find you end up migrating emails that are actually shortcuts (aka stubs) to archived emails.

In our experience – as long as you have a strategy for eventually removing these shortcuts and replacing them with the original archived item – this is a workable solution.

* If shortcuts moved into Office 365 don’t work ‘out of the box’ for your legacy archive we can help you make it work!

2. Being Selective Can Really Pay Off

With the GDPR top of mind, a migration is a good point to review what you’re retaining in your archive.

A simple strategy is to exclude anything that’s beyond your retention policy but it’s possible to be more ‘fine-grained’ than this.

Although it’s possible to start applying retention policies once you’ve migrated your archives into Office 365, deleting data pre-migration can save both migration time and costs.

3. Not Everything Has a ‘Home’ in Office 365

As you migrate to Office 365, making the decision about what to do with legacy emails – especially those belonging to ‘leavers’, can be challenging.  This is especially the case for organisations that have lots of ‘churn’ or have ‘orphaned’ email records going back many years.

Although it’s possible to migrate a leaver’s data into an Office 365 mailbox that has been placed on Litigation Hold, delete it and then recycle the licence after 90 days (using the inactive mailbox service in Office 365), many of the customers we work with seek alternative, and frankly, much faster and lower-cost solutions.

This includes migrating leavers (and other data items that don’t fit conveniently into Office 365) into HubStor for Azure, which gives low-cost storage, retention management and eDiscovery.

4. Have a Plan for Killing Off Your On-prem Archive

Following successful and verified migration of your archives to Office 365, it’s best practice to decommission your on-premises archive and delete all the archive databases and storage.

Any email records that you’ve ‘policy excluded’ from a migration (see point 2 above) should also be deleted.

TIP: Make sure any backups, test systems or copies of the databases are hunted down and removed. 

In the event of an eDiscovery, ‘viable’ copies of an archive you thought were long gone could crawl out of the woodwork and become part of a legal case.  They could also be found (inadvertently or otherwise) on your network or old servers, and again, could compromise your compliance with the GDPR.

5. Think Long & Hard About Your Journal Migration Strategy

There’s a lot of debate around what to do with journals (including journal archives) when moving to Office 365.

Whilst it’s possible to take an existing journal and ‘de-single-instance’ its contents for migration into individual Office 365 user mailboxes – the process can be quite lengthy.

The only issue is that any other route (such as migrating chunks of a journal into multiple individual mailboxes in Office 365) can bite you in the rear.  For example:

  1. Using In-Place hold on single mailbox containing emails belonging to multiple custodians breaks Microsoft’s own licencing rules
  2. Creating a bunch of shared custodian mailboxes will give you an eDiscovery headache further down the track (because the person doing the search won’t know which mailboxes to include – and could easily exclude them)

Migrate your archives into Microsoft 365

If you want to drill into the options and trade-offs when migrating your journal (or talk about any of the other points above) get in touch.

Get in touch

U-turn on inactive mailbox charges

Posted on by Essential Marketing

Free retention of ex-employee’s data

At present, if you want to retain the mailbox contents of former employees’ mailboxes there’s a facility called ‘Inactive Mailboxes’ that you can use.  The great thing is that if you follow Microsoft’s steps, you can re-use the licence associated with the ex-employees mailbox for someone else, so effectively there’s currently no charge for this facility.

Watch this space, however, as back in 2017 Microsoft was on the verge of introducing a charge for inactive mailboxes, and it’s our guess they could consider doing it again.

Inactive mailboxes could be chargeable…

Back in late 2017, Microsoft was on the verge of charging for inactive Office 365 mailbox licences.  It’s our prediction that this could happen again.

At the time, Microsoft faced a backlash from their customers and MVPs during Ignite 2017, and did a U-turn on charges for inactive mailboxes.

https://www.petri.com/no-licenses-office-365-inactive-mailboxes

Having seen the proposed licence plans, we’re not surprised it caused a stir. Inactive mailboxes represent a significant volume of data.

“When we do an analysis scan before moving email archives to Office 365, it’s not unusual for about 70% of the contents to belong to ex-employees” Annie Holder, Migration Consultant

The U-turn highlighted the demands that businesses are making on Microsoft to support proper governance of their email and other data.  Right now, the way Microsoft 365 helps you manage the full lifecycle and eDiscovery of email is impressive.

We will, however, watch with interest how Microsoft adapts to accommodating the vast churn of mailboxes from a licencing perspective.

Not just because of the potential future cost of retaining sheer volumes of it, but also because of a greater responsibility to keep it secure, minimise the risk it represents and fulfil obligations around data protection.

Managing data that doesn’t have ‘an obvious home’

Handling the retention of leaver’s mailboxes, SharePoint and OneDrives is sometimes still only part of the story.

Many cloud project teams are now turning their attention to other more complicated stores of data – like legacy Journals, public folders, PST files, file shares… data that sometimes doesn’t seem to have an obvious home in the Microsoft cloud.

Retention of content that doesn’t fit neatly into Office 365 (such as legacy data on file servers), is a topic we regularly address with our customers.

Inactive mailbox charges

If you have legacy on-premises content you want to preserve and do eDiscovery on, but you’re not sure where to start, get in touch.

Get in touch

Why BCC is important when you migrate your journals to Office 365

Posted on by Essential Marketing

Why are BCC’d recipients so important?

In relation to email, BCC stands for “blind carbon copy.” Just like CC, BCC is a way of sending copies of an email to other people. The difference is that recipients CC’d on an email have no visibility of the fact that other people may have also received the same email.

I think we’ve all been on the receiving end of a marketing email that’s been inadvertently sent to CC’d a circulation list.  This is where BCC comes into its own, but there’s other scenarios where BCC is used.

A key thing to consider is “Why do people use BCC in work-related emails”?

  • To raise an issue concerning a co-worker?
  • To lodge a confidential record of an email exchange with a third-party?

Arguably the use of BCC is secretive and deceptive and it follows that the nature of the email will be more ‘shady’ or confidential than an openly CC’d email.  It also follows that the person being BCC’d is just as important, if not more so, than those that are CC’d.

The good news:

The default Exchange journal setting (and that of most hosted journaling services such as Mimecast) is called an ‘envelope’ journal.  The envelope includes a record of the TO: and CC: fields as well as any BCC’d recipients and all the individuals included in your local distribution lists (DL) at the point in time the email was received by your messaging transport agent (MTA).

The bad news:

In the process of migrating to Office 365, you could be stripping out BCC and DL information from your email records.

Having helped with extremely large corporate email investigations, we know the importance of maintaining complete email records and maintaining due diligence when handling email archives in particular.  https://www.theguardian.com/media/2011/jul/08/phone-hacking-emails-news-international

What’s the problem with Office 365 & Journaling?

The key ‘gotcha’ is that Office 365 does not have a journal service – at all. 

Until recently if you wanted to move to Office 365 and maintain a conventional envelope journal you’d have had to subscribe to a third-party service from an organisation like Mimecast, or keep an Exchange journal running back on-premises. 

But in the last few years Microsoft has been filling a few holes.  Office 365 can now effectively replace the role of the envelope journal and provide a one-stop-shop for compliant and complete email records retention.  This is how it works:

  • Instead of using a large, centralised, single-instanced mailbox that is inherently difficult to scale and failover, Microsoft uses its optimised multi-instance storage model.  This allows each user to retain his/her copy (journal) of all emails sent/received with zero performance penalty and no single point of failure.
  • By putting all relevant mailboxes on In-Place Hold, all emails sent and received are retained indefinitely.
  • Deleted emails are removed from the user’s view, but held into a special hidden folder inside the Recoverable Items Folder (RIF), where they are available to the eDiscovery process.
  • Any BCC’d recipients will be retained indefinitely in the senders’ mailboxes.
  • The members of any distribution lists (DLs) are expanded at the point of sending and stored in hidden headers in senders’ emails so they are fully discoverable.
  • Ex-employee’s mailboxes (i.e. those belonging to leavers) can be put on Indefinite Hold and made available for eDiscovery, without a license penalty (using Microsoft’s inactive mailbox service).

So assuming you’re not going to dump over 10 years’ worth of email records when you move, all you’ve got to do it map what’s in your existing journals and any journal archives (which are commonplace given the size to which journals can grow) into the new model.

You’ve actually got a few options for doing this, ranging from quick and potentially dirty to slower and comprehensive?

Email Journal Migration

Want to get the full scoop on how it all works?  Get in touch today.

Get in touch

Size Is Important When You Migrate eMail to Office 365

Posted on by David Kellett

Apart from ensuring you meet the terms of your Microsoft 365 licence plan, understanding your overall mailbox sizes – as well as the size of any archives you want to migrate – can help you when migrating to Office 365 by:

  • ensuring the optimum end user experience,
  • minimising network impact, and
  • getting a handle on just how long it’s going to take you to complete your migration – something we are always asked to help estimate

The amount you have to move, where it resides at present, and where you want to put it as you move it into Office 365, can also shape your migration strategy.

Apologies in advance, as this has turned into a longer article than we originally intended – but it’s a big subject! 

Dealing with Large Mailboxes when Moving to Office 365

The first thing you’ll probably want to do is pinpoint larger mailboxes.  Apart from taking a long time to get moved across, if you have any mailboxes that exceed 100 GB (or more realistically, 90GB) you will need to adopt a different strategy for moving these mailboxes.

Suggesting that users start deleting (or centrally enforcing deletions) to get back under 90GBs is an approach that just won’t wash these days…so what are the options?

A common approach taken by our customers is to archive larger mailboxes to bring them down in size.  This can be achieved:

  • using an existing 3rd-party archive or
  • using native Exchange archiving.

If you plan to use Exchange archiving, here’s a neat script from our colleague Michael Van Determining the effect of Archive-policies Mailbox Archive SizeHorenbeeck to help you estimate the impact of a given set of Exchange retention policies on mailbox sizes – e.g. Items over a certain Age Limit.

http://vanhybrid.com/2013/12/30/estimating-the-size-of-an-exchange-online-archive/

See the bottom of this article for tools to help with generating these reports.

Watch out for Large Attachments

You can now migrate messages up to 150MB in size to Office 365, but this means you still need to weed out those odd few messages that are bigger than this prior to migration (otherwise they won’t migrate) and establish your game plan for moving them.

Is Microsoft’s On-line Archive Really Unlimited?

Over the years Microsoft has shifted its stance on how big archive mailboxes can grow.

According to the current service description, a 100GB limit is initially applied and when a feature called auto-expanding is enabled, additional storage is automatically added, without a mention of an upper limit*.

There are, however, caveats to be aware of, including:

  • the rate at which archive mailboxes can grow must not exceed more than 1GB a day
  • archives cannot be used for the purposes of journalling
  • Microsoft can deny the use of archives to store email belonging to users other than the individual the mailbox is licenced for (although shared mailbox archives are supported, which is a little confusing).

*Microsoft currently uses the term ‘unlimited archiving’ (previously a 1TB limit was imposed).  This is potentially a huge amount of storage to offer so our advice is to anticipate a limit from Microsoft at some time in the future.  We may see a limit introduced hand-in-hand with the provision of administrative tools for managing storage.

Estimating Your Optimum Office 365 Migration Batch Size

Having a profile of your overall mailbox and archive sizes will help you establish a throughput baseline, build appropriately-sized mailbox batches, and help gauge how long your migration to Office 365 will take ‘end-to-end’.

If you’re migrating in a hybrid environment, the usual ‘modus operandi’ is to schedule MRS (or other migration tool you might be using) to run outside of core office hours so you can push through as many mailboxes as possible – as quickly as possible – without incurring the wrath of end users.  Even more critical, if you’re migrating using a cutover or staged approach, you’ll want to work out if you can meet desired timelines (e.g. over a Bank Holiday weekend).

Watch out though –  it can be  slower during out-of-hours and the weekend, because your are not the only one to have the idea to schedule big moves during these times :-).

If you trawl the Internet you’ll get a whole bunch of different Office 365 ingestion rates ranging from 1GB per hour to over a TB a day.  Our best advice is to carry out test migrations in your own environment – with a reasonably large data set – to work out the maximum number and size of mailboxes you can move in a given time-frame.

The more mailboxes you can move in parallel the better, but simply cranking things up doesn’t mean faster migrations as you’ll find you hit a ceiling..

Even though you may be able to move data into Office 365 at a rate of knots, the fact is you are pretty much powerless over the speed at which Office 365 can ingest what you send it.

Here’s a useful article by Steve Goodman to help pinpoint where any bottlenecks might be and determine whether you can do anything about it http://searchexchange.techtarget.com/tip/Simple-ways-to-troubleshoot-slow-mailbox-moves-to-Office-365

Another factor has proven to be the physical location of your site in relation to your target Office 365 service.

Microsoft articles to help you diagnose performance issues depending on your region can be found here:

How an ‘Archive First’ Approach Can Expedite Your Move to Office 365

If you’re migrating in a hybrid environment (and have purchased the relevant plan), you can take advantage of the fact you can provision hosted In-Place Archives ahead of moving primary mailboxes to the cloud.

This offers several benefits, for example

  • You get to migrate email that is less mission-critical first.  If something goes wrong or if it takes longer than you’d hoped, you haven’t impacted primary mailboxes
  • You can ‘crank up’ your existing archiving policy to minimise the size (and migration times) of your ‘live’ primary mailboxes
  • You can move archived emails separately from primary mailboxes
  • 3rd-party archive migration tools can be faster – for example, our migration service has benchmarked 3x faster (at least) than MRS

A few pointers here:

  • If you use native Exchange archiving, you’ll need to check you have enough temporary space on your Exchange server to ‘pre-stage’ the archived emails. Customers that have taken this approach have been able to release the space on their Exchange servers for re-use as they move onto the next batch. NB Mailscape reports are available to help you check that white space and storage is maintained within acceptable operating limits.
  • Note that you could, in theory, create Exchange archive policies that move items direct from on-premises into hosted In-Place Archives, but it’s worth comparing the two approaches to see which will work best for you.  I.e.:
    • Archive on-premises, and then use MRS to move to Office 365 In-Place Archives
    • Migrate from primary mailboxes direct to Office 365 In-Place Archives

Michael Van Horenbeeck prefers to create on-premises archives first and then move them later using MRS.  This is because of how Outlook interacts with the archive and the effect the initial ‘seeding’ of the archive has.  His script (referred to earlier) will help you determine the amount of data that will need to pass through your internet connection which ever route you take.

Using 3rd-party Tools to Speed Up Your Move to Office 365

We’ve already spoken about using 3rd-party migration tools to move the contents of 3rd-party archives, but the concept of using tools to help moving primary Exchange mailboxes is a well-trodden path.

Tools are available from BinaryTree, Priasoft, BitTitan, Dell (Quest) and more.  Apart from typically being faster than using Microsoft tools*, useful things to look out for that will help with your migration (particularly when you have large mailboxes to handle) include:

  • The ability to populate online Primary mailboxes with say, the first 3 months’ of emails (to get users on-boarded REALLY quickly) and then gradually ‘backfill’ the remaining email over time.
  • The ability to migrate selected items from the Primary mailbox to the Office 365 Primary OR In-Place Archive depending on policies.  E.g.:
    • Move everything < 2 years to the online Primary mailbox
    • Move everything > 2 years to the online In-Place archive
  • The ability to correctly handle any legacy archive shortcuts as they get encountered during the migration.  We recently encountered a widely-used 3rd-party solution that simply migrated shortcuts into Office 365 and actually ‘broke’ them in the process, making it impossible to identify and manage them ‘post-migration’.    

*Generally speaking, an approach that uses the EWS protocol to talk to Office 365 (which many 3rd-party migration tools use) as opposed to MAPI (which MRS uses) is faster.  This is  largely owing to the fact that the ability to use impersonation enables you to move more mailboxes concurrently.

Watch out for ‘Archive Expansion’

If you’re migrating items that are currently stored in a 3rd-party archive, bear in mind that you’ll have an expansion factor.

100GB of archived data could equate to 10 times this amount in Office 365, as you are:

  • re-hydrating the original item as it gets moved into to Office 365
  • losing the effects of single-instance-storage and compression that you got with archiving

Contact Essential if you’d like assistance in estimating what your archived data might equate to as you migrate to Office 365.

Beware Large Mailboxes & OST Sync Post-Migration

If you think that 50GB is plenty of space (and as a result, you elect not to purchase a plan that includes In-Place Archives), bear in mind this ‘gotcha’:

Even though primary mailbox sizes in Office 365 are generous, if you plan to use ‘full Outlook’ (desktop) clients with the ability to access emails when working offline (i.e. using OST files), you could encounter problems.  This is because having overly large online mailboxes to synchronise onto local systems will create a huge network overhead during the migration process.

If you use Outlook client 2013 or above there’s a slider to control how much gets synchronised, but older versions of Outlook may struggle to provide a performant synchronisation service and could literally kill your network as mailboxes get switched over to Office 365.

Conclusion: There’s More Than One Way to ‘Skin the Migration Cat’

Depending on your overall migration strategy (e.g. cut-over, staged, hybrid) and what you have to migrate, which could include large mailboxes, archives, public folders, leavers’ mailboxes and journal mailboxes (these are a whole different subject which we’ll cover in later articles), your migration could be managed in a number of different ways to mitigate risk and ensure minimal downtime for end users.

There’s no ‘one size fits all’.

Essential has supported customer migrations that have used a variety of techniques to meet technical and business needs.  

We are also flexible in our approach, and happy to help your own IT team oversee the migration, or work alongside a chosen on-boarding partner to address the trickier bits, like migrating 3rd-party archives and distributed PSTs.

Essential has supported customer migrations that have used a variety of techniques to meet technical and business needs.  

Call today to talk about your plans and concerns and what we can do to help.

Get in touch

Why you need to fix mis-matched UPNs and SMTP addresses before you migrate to Office 365

Posted on by Essential Marketing

So what’s the problem?

When you log on to Office 365 you’re functioning in the ‘big wide world’, and Microsoft wants to know who you are.

This means that logon IDs have to include an externally valid domain (the one you have registered and verified within Office 365) along with a unique username within that domain.

By default, Office 365 uses what’s called the User Principal Name (aka the UPN) to achieve this.

A UPN is constructed very much like a user’s email SMTP address, taking the format:

username@domain

If you take a look at your current on-premises UPN (labelled as the ‘User logon name:’ in the Active Directory Properties View) you’ll see it comprises:

  • a username, which could take a number of forms.  For example:
    • John.Smith OR
    • SaraA (this being a format typical of the older pre-Windows 2000 DomainUsername authentication aka samAccountName)
  • a UPN ‘suffix’ (i.e. domain) which in this case, is set to HQ.domain.com

AD properties SMTP UPN

So the first thing you need to do when you migrate to Office 365 is to check that you have a UPN suffix that matches in with the external domain you’ll be using for Office 365.  Any internal routing names such as HQ and ‘local’ mean nothing to Office 365.

In this example, HQ.domain.com would become domain.com – something that would be easy to fix with the Active Directory module in PowerShell.

What about what’s to the left of the @ sign?  

As you can see from the examples, you may have a mix of different username formats.  This can depend on when the users were added and the conventions of the current administrator.  You might also synchronise usernames and other attributes in from another application, such as an HR system.

The fact is, any of these username formats are valid as part of a UPN as long as:

  1. they don’t contain invalid characters
  2. they are unique
  3. users know what their UPN is

In our example, Sara would logon as SaraA@domain.com and John would logon as John.Smith@domain.com(which just so happens to be his email address).

But herein lies the problem.  Because the Office 365 UPN is formatted like an email address, and for some, but not all your users, it may actually be their email address, confusion reigns supreme. 

This confusion is compounded by the fact that, for example, the Activation window for Office Pro Plus prompts the user to enter their email address.  Likewise the iOS versions of Office and OneDrive prompt for email address.  In these examples Microsoft and Apple really mean ‘UPN’ and not ’email address’.

The upshot of all of this is that, although your UPN and SMTP email address aren’t necessarily one and the same, when you migrate to Office 365 you are best advised to make UPNs match users’ SMTP addresses.

That way, you’ll save a heck of a lot of support calls.

HOW TO FIX STICKY BIT 1:  Make UPN=SMTP

The tricky element is identifying non-matching UPNs and SMTPs and with organisations expanding every day, the number of occurrences can continue to increase.

There’s some useful articles by Joe Palarchio and Steve Goodman that explain more on this subject.

One final note:  Although it is also possible to set up what’s called an Alternate Login ID to cover for the fact that users will get confused, or that you may have an umbrella Office 365 domain for multiple subsidiaries with different email addresses, or you have legacy applications that mean you cannot change the UPN, there are caveats to using this feature – especially if you have a hybrid environment.

Migrate your archives into Microsoft 365 & other cloud services.

Start your FREE Trial

Get a Quote

Is Microsoft 365 a one-stop-shop for your email compliance?

Posted on by Essential Marketing

Discover How (and why) Microsoft 365 Replaces The ‘Traditional’ Email Journaling Service.

Have you ever wondered why Microsoft 365 doesn’t provide a ‘native’ email journaling service (like your old on-premises Exchange server used to).

  • Do you still need to use a third-party journaling service (such as Mimecast or Proofpoint) or an on-premises Exchange server?
  • If not, how is Microsoft now ‘filling the journal gap’
  • What you need to do to migrate an existing on-premises journal or cloud journal into the new ‘Microsoft way of doing things’?

This white paper addresses all these questions and more. 

Download your copy of the Making Office 365 One-Stop-Shop for Email Records Compliance white paper.

Discover How (and why) Microsoft 365 Replaces The ‘Traditional’ Email Journaling Service

Get in touch to find out more about your options for handling your legacy email Journal when you use Microsoft 365.

Get in touch

Related Blog Articles

ARTICLE

Top 10 – Reasons People Want to Move Archives Around

Our archive migration service is taking off and we are getting a reputation for being the 'Pickfords' Removals' of the email archive world. Each new project brings its own set [...]

Read More

ARTICLE

Computing magazine article featuring Essential

Prompted by the press that surrounded the email audit trails at News International, Computing magazine interviewed Essential for its views on the importance of strong email and data governance in [...]

Read More

ARTICLE

Email Free Days

Mindful of the fact that sending an email to a colleague can sometimes be a poor substitute for a telephone conversation or a face-to-face meeting, RCT Homes has introduced the [...]

Read More
?>

Sign up to our newsletter today