Rate limiting is a mechanism that most cloud services apply to ensure there are adequate system resources for the web app they provide to their end users, and to ensure that data is protected from malicious acts such as DOS attacks. Unfortunately, it can significantly slow down a migration. Some steps you can take to mitigate rate limiting are:
- Before migration starts, notify your Tervela sales rep if the migration is more than 10TB. Tervela sales will work with Microsoft to ensure that your SharePoint account is fully provisioned with all of the resources it needs for the migration. Tervela sales will need the tenant name and tenant ID to forward to Microsoft.
- Run migrations after normal business hours for where your SharePoint data is stored. If the data is stored in the Pacific time zone, run jobs after 6PM PT and on weekends.
- If the migration is large, split it up into several jobs. Make sure that each job has an SPO system with a different oauth user. For best results, you will also need five different oauth users for the source, so you don’t get rate limiting from the source.
- Ensure that no other backup utilities, syncs, DLPs, migration tools, or anything else is running on your site that is imposing a large request load on the site.
- If you are seeing retry rates that are greater than 30% on your job, contact Tervela. In some cases, we will ask you to file a ticket with Microsoft support to investigate. For fastest service, send the ticket number to firstname.lastname@example.org. Following up on the ticket is still the customer’s responsibility, but we can sometimes expedite resolution if we know the ticket number.
- Batch retries (i.e., the “JobImportant” errors seen in the event log) are Microsoft errors uploading content from Azure to Sharepoint. They are entirely separate from the retry rates seen in the job metrics balloon, which typically are due to 429 errors. Batch retries happen after CFP has transferred the data to Azure on the tenant, and CFP has no control over these batch retries.