Key Considerations of Data Migration – Part 2: Dealing with Complicated Metadata

In the first post of this series “Know your Network, Scope your Data, Keep it Safe”, we described how understanding the capacity of your network, how much data you will be migrating, and how you plan to keep it safe are three crucial elements in planning for a mass data migration. In this post we will summarize what you need to know before attempting to map over metadata to a new file sync and share (EFSS) platform.

So you’ve been selected to migrate tens, hundreds, or thousands of users to a new EFSS solution. Of course, like anything else, the users expect everything to work perfectly – exactly how it was in the old system (plus more). Ensuring that critical metadata reaches the new cloud EFSS will reduce friction for your users and save you from incessant whining once they adopt the new system. It is important that a migration plan includes appropriate expectations for what kind of metadata can be transferred to the new platform, and how that process will actually occur.

Metadata is an all encompassing and fairly annoying term. It could mean … anything. So we will narrow metadata down as to it applies to typical files. At Cloud FastPath, we typically encounter three main groups:

Standard metadata – file created date, file size, last modified date, file name, file type.
Ownership metadata – user and group permissions of files or directories
Custom and taxonomy metadata – associated with filetypes or taxonomy

Important aspects to consider about transferring standard metadata:

There are a few cloud file sync and share platforms which automatically reset all or most standard metadata! For instance, all of your file’s created date would get reset to the date they were moved to the new platform. Why? Who knows. Without proper intervention this would spell disaster for your users post migration.

There are also a variety of restrictions in terms of file name and file type. Many cloud file sync and shares will not allow or poorly handle certain encoding or characters (a test: try uploading a file titled “myㅕ%2–0PDF&E384B4%20file%23naㅅme” to any platform) and limit filenames to a certain character size. In addition, cloud providers typically have restrictions on what types of files can be uploaded into their platform. For example .iso, .exe, and .filetype:$DATA. Without systems to handle strange characters, automatically limit filenames, or provide alerting for disallowed files, many files won’t make it to the destination. And even worse, you won’t know what files didn’t make it until a user comes knocking on your door blaming you for losing their precious marketing images.

hand-895588_640

very important marketing image example #success

Important aspects to consider about transferring ownership metadata:

A migration to a new storage platform will be difficult for your users and of course they don’t care about the effort you’ve put into it. They expect to have the same access to the same files and folders as before.

Depending on the number of users being migrated, transferring user and group permissions can be incredibly time consuming, frustrating, and can outright prevent a successful migration.

Cloud file sync and share providers have different approaches to how they treat permissions. While some have identical waterfall-style or relational permission structures like that of Windows fileservers, many do not. It may seem like moving waterfall permissions from one system to another would be fairly straightforward, but it isn’t. The tooling that is available for typical on-premises systems is not available for web based file sync and share platforms. Therefore, you have to create, test, maintain, and test again a custom mapping program that integrates the destination’s API with on-premises systems.

Consider a migration with 500 users. Each user has an average of 30 folders and in each has an average of 50 files. If there are mapping mismatches for let’s say only 20 users, that means there would be 30,000 errors that need to be manually corrected.

15klx1

User driven migration is sometimes an option to retain file permissions. However, these style of migrations usually end up in a horrible hybrid-purgatory-hell situation, where some users have fully migrated, some have migrated some of their content, and some not at all.

Learn how to migrate user and group permissions with Cloud FastPath.

Important aspects to consider about transferring custom metadata:

Custom metadata usually originates from proprietary ECM platforms like SharePoint, Documentum, Filenet, Alfresco, and the like. These are feature rich platforms which offer essentially unlimited customization. Most custom metadata in these platforms is particular to a filetype such as EXIF with digital photos, or typical taxonomy-like tags and categories.

The biggest issue with transferring custom metadata is the extent of how much it is supported in a new EFSS solution. Among the major current EFSS providers a few have just begun to create features around custom metadata. SharePoint Online is the most robust platform in terms of custom metadata, and most would be supported. However, migrating custom metadata from non SharePoint systems into SharePoint Online will also require similar mapping systems as permission mapping. For now, migrating custom metadata will be a very manual process and it may be best to use EFSS functionality to replace it or leave it behind.

In conclusion

Before selecting a new cloud EFSS the transfer of metadata must be planned out. Ensure the new platform can support the metadata you want migrated. If it can’t, communicate this to admins and users. During the migration planning and execution process use tools or create systems to limit the friction on your users – the new cloud features shouldn’t be overshadowed by a limited migration.

For a full guide on how to migrate to EFSS and cloud file servers, get our ebook:
Data Migration Fundamentals