DESCRIPTION: OneDrive is a cloud storage that works in conjunction with a Microsoft account, frequently with a Windows based product. The business version, OneDrive for Business, supports OneDrive in a more collaborative multi-user environment. To access all user accounts effectively, you should provide global admin credentials for this system to log in to your OneDrive account. Use the Administrator Account version of the OneDrive for Business system to run account mapping jobs.
RECOMMENDED POPS: Azure.
What’s New: CFP has added advanced permissions to OneDrive, and there is a new role, Restricted Viewer, that is listed in the mapping spreadsheet. Restricted Viewers have view rights but no download rights. Along with this addition, Write (Users) and Write (Groups) will now be called Contribute (Users) and Contribute (Groups) respectively to be consistent with Microsoft terminology in the Advanced Permissions settings.
Identify Paths, Folder and File Names That Are Too Long
Two of the most common reasons for file transfer errors during OneDrive migrations are ‘Name too long’ and ‘Path too long.’ OneDrive has a limit of 255 characters for any file or folder, and a limit of 400 characters for the full URL path to the item. The URL includes the full OneDrive site name. So if the path for your document on OneDrive is this:
The path length calculation is based on this string:
Note that characters such as spaces, slashes, ampersands and unicode characters will all be converted in URL encoding, and the resulting encoding for a single character may be several characters in length.
If you have not named your OneDrive tenant before starting the migration, it is immensely helpful to give it a short name to minimize the incidence of path length errors. If you are moving your data to a specific folder on OneDrive, keeping the folder name short will likewise minimize errors and the need to rename paths to a smaller length. Renaming or restructuring paths is not advised once the migration has started, as duplicate data sets may result.
Running a simulation prior to the actual data transfer will identify paths, folder names and file names that are too long for OneDrive. You can then adjust the paths prior to running the actual migration. Path renaming done after the map is generated can result in mapping errors and other issues with your migration, so re-running the map after renaming is recommended. Typically, data that has paths too long is clustered into a few folders, and is reasonably easy to address. Contact firstname.lastname@example.org for advice on whether you need to re-run a map. Simulation will also detect files greater than 15GB, which is the limit for OneDrive.
You can get a rough idea of paths that are too long by running an analysis at the very outset of the project and then running a (LEN) function in Excel for all of the paths in the analysis report. Any character count that is > ~250 characters or so is in the risk zone for path too long errors.
Many OneNote notebooks, and the associated items such as Notebook packages, may not transfer from or to OneDrive due to the facts that they are highly dependent on the ecosystem in which they were created, and they do not always have well defined properties as files. If they do transfer, they may not open in the same state as on the source, for reasons independent of CFP. Please make other provisions to transfer these items.
Pre-provision All Target User Accounts
Tervela cannot move data to a OneDrive account unless you have allocated storage space for it. To provision accounts in bulk, refer to Microsoft’s documentation at https://docs.microsoft.com/en-us/onedrive/pre-provision-accounts.
Turning Off OneDrive in Office365
You can hide OneDrive and Sharepoint as options for saving Microsoft files to prevent users from saving documents to OneDrive until after migration is complete. Refer to Microsoft’s documentation https://docs.microsoft.com/en-us/sharepoint/hide-app-tiles-on-app-launcher.
Global Admin Selection
Microsoft’s api makes the oauth user an owner for all data transferred to OneDrive and SharePoint sites. For those users who do not want the oauth to have this access, create a global admin specifically for the migration. Once the migration is complete, delete the oauth user, and they will be concurrently deleted as an owner for all migrated content.
Email Notifications for Shared Items
Internal Sharees: Cloud FastPath suppresses sharing notifications automatically for internal sharees.
External Sharees: External sharees will receive an email notification when an item is shared with them on a given tenant. The email is a security measure to verify the user’s identity, but it also is a means to provide access to the shared item(s). Once the external sharee clicks on a link in the email to accept the share, they will be registered with the tenant as an external user with a user id on that tenant. At that point, all further emails to that user will be suppressed.
If you don’t see your external sharees listed in OneDrive: Make sure the external sharee has responded to their invitation to the folder. They won’t have access to the folder until they do so. In some instances, an external sharee will not be displayed in the Manage Access panel for the folder or file. Click on the Advanced link at the bottom of the panel, and you will call up a Permissions tab that lists Limited Access users. You should see the external sharee listed as a Limited Access user.
A very helpful article in understanding some of the ACL limits in SPO/ODFB is https://docs.microsoft.com/en-us/office365/servicedescriptions/sharepoint-online-service-description/sharepoint-online-limits#items-in-lists-and-libraries. There are two types of ACL limits on Microsoft platforms:
- ACL quotas
- Number of items that have a given ACL
OneDrive for Business has a limit of 50,000 ACLs per personal site, or account. For best performance on OneDrive after migration completes, Microsoft recommends that you stay below 5000 ACLs per account, and, in many instances, a fraction of this number. The number of ACLs that will be applied by CFP is roughly equivalent to the number of lines in the Paths tab of the spreadsheet – but the specifics are:
- each item (file or folder) that is shared counts as one ACL, regardless of the number of users and groups that are sharees for that item
- if a folder is shared, that counts as one ACL, but any child items that inherit permissions from the folder do not count against the ACL quota
Number of Items That Have a Given ACL
The maximum number of items that can have a given ACL is 100,000. What that means is that if you have a folder that contains more than 100,000 total files and folders (total items, and not just the immediate children), adding an ACL to the top level folder will fail because it exceeds Microsoft’s limit.
If you identify a folder that has more than 100,000 items, here are some things you can do to address the limitation.
- Long term, the best strategy is to break the folder up into smaller pieces, and migrate it as a set of smaller folders. email@example.com can show you the best way to do this. This has the added benefit of decreasing load times for the folder contents in the ODFB web app once migration has completed.
- Deselect the Advanced Option “Keep Parent Permissions” to prevent the site permissions from inheriting down to the folder. You’ll need to add those permissions back for individual subfolders of the main folder so that the site users have access to it – and make sure that those subfolders each have less than 100,000 items. Move the data over in one pass, and apply the ACLs in a separate pass.
- Keep sharing strategies as simple as possible. Reduced permissions on large subfolders such as these – where one or two items have reduced permissions and the other items have the same permissions as the parent – can result in exceeding the 50,000 total ACL quota.
Rate Limiting, and How to Avoid It
Rate limiting is a mechanism that most cloud services apply to ensure there are adequate system resources for the web app they provide to their end users, and to ensure that data is protected from malicious acts such as DOS attacks. Unfortunately, it can significantly slow down a migration. Some steps you can take to mitigate rate limiting are:
- Before migration starts, notify your Tervela sales rep if the migration is more than 10TB. Tervela sales will work with Microsoft to ensure that your OneDrive account is fully provisioned with all of the resources it needs for the migration. Tervela sales will need the tenant name and tenant ID to forward to Microsoft.
- Run migrations after normal business hours for where your OneDrive data is stored. If the data is stored in the Pacific time zone, run jobs after 6PM PT and on weekends.
- If the migration is large, split it up into several jobs. Make sure that each job has a OneDrive system with a different oauth user. That means that if you want to run five concurrent jobs, you will need five global admin users. For best results, you will also need five different oauth users for the source, so you don’t get rate limiting from the source.
- Make sure your OneDrive system credentials were entered after August 1, 2018. Tervela has made significant enhancements to the OneDrive adapter, but you will need to enter credentials after the above date to take advantage of them.
- Ensure that no other backup utilities, syncs, DLPs, migration tools, or anything else is running on your site that is imposing a large request load on the site.
- If you are seeing retry rates that are greater than 30% on your job, contact Tervela. In some cases, we will ask you to file a ticket with Microsoft support to investigate. For fastest service, send the ticket number to firstname.lastname@example.org. Following up on the ticket is still the customer’s responsibility, but we can sometimes expedite resolution if we know the ticket number.
- Batch retries (i.e., the “JobImportant” errors seen in the event log) are Microsoft errors uploading content from Azure to Sharepoint. They are entirely separate from the retry rates seen in the job metrics balloon, which typically are due to 429 errors. Batch retries happen after CFP has transferred the data to Azure on the tenant, and CFP has no control over these batch retries.
ONEDRIVE SITE URL Enter the URL for your OneDrive site. Usually this will be in the format https://companyname-my.sharepoint.com. For additional information on OneDrive Site URL’s, see https://support.office.com/en-gb/article/About-your-initial-onmicrosoft-com-domain-in-Office-365-b9fc3018-8844-43f3-8db1-1b3a8e9cfd5a.
TENANT URL: Enter your O365 Tenant URL (e.g., https://companyName-admin.sharepoint.com). This field is primarily for users who have custom URLs for their tenant that deviate from the standard format. Leave this field blank unless advised to complete it by Tervela support.
OneDrive Source and Target Node Configuration
Source Node Configuration
SOURCE ACCOUNT LIST Select the account(s) you wish to transfer. If you select multiple accounts, you will transfer the entire contents of each account. If you select a single account, you will see an additional field, Source Files and Folders. You can select all of the data from the one account using the Source Files and Folders selector, or you can select a subset of the data in the one account.
SOURCE FOLDER/LIST NAME This field will only be displayed if a single account is selected from the Source Account List chooser. If you receive an error ‘User has no Edit permission on site <sitename>,’ please see instructions below for adding the admin user as a site collection owner.
Target Node Configuration
TARGET ACCOUNT NAME The account that will receive all files from the source. Note that if you select a team member, it will nullify any directives from the spreadsheet for account mapping. If you are running account mapping, this field should be left blank.
DESTINATION FOLDER Enter the folder name into which you wish to transfer your files on the target (destination) server. If you receive an error ‘User has no Edit permission on site <sitename>,’ please see instructions below for adding the admin user as a site collection owner.
Account Mapping Considerations: Adding Redirects to SharePoint
An account mapping job that has a OneDrive target can have redirects that send data to SharePoint sites/doc libraries. The formats are as follows:
- To send data to a document library that is directly off the root, enter /:/docLibraryName, where the document library is docLibraryName.
- To send data to a subsite/doc library, enter the /subsite path:/document library path. Example: /:sites/engineering/Documents goes to the doc library Documents in the /sites/engineering subsite. /:marketing/DocLib/Folder1 goes to the folder Folder1 in the /marketing/DocLib subsite/document library.
- To redirect to a different OneDrive account, simply enter the account to which you want to redirect. Do not use path redirects.
- The formatting rules for redirect paths in SharePoint jobs are different from OneDrive. Use the correct path conventions for your job’s target.
- Always use the Document Library names specified in the left-hand sidebar of the site, and not what displays in the browser’s URL window.
Microsoft conventionally changes the default document library name, “Documents,” to “Shared%20Documents” in the URL. Use “Documents” when constructing paths. Likewise, if you have a root document library “/doc lib with spaces,” enter “/:/doc lib with spaces,” not “/:/doc%20lib%20with%20spaces..”
- “Forms” is a reserved word in SPO. You cannot write to /Forms subsites. You can create – and transfer – folders called “Forms,” but CFP will convert them to the UTF equivalent name (_u0066__u006f__u0072__u006d__u0073_) on the SPO target if they are the top folder in the subsite. Folders further down – e.g., /sites/subsite/docLib/FolderA/Forms – will transfer as is.