Key Considerations of Data Migration – Part 1: Know your Network, Scope your Data, Keep it Safe

The bulk of the time required to perform a successful cloud file transfer is normally thought of as the actual transfer. Internet bandwidth, complicated filesets, and error handling certainly do extend the transfer time, however, with careful planning these aspects will have less of an effect. While planning a data migration, three aspects of the planning process correlate directly to the transfer time and overall success: understanding network limitations, properly scoping the content, and keeping data secure.

Understanding your network

Normally the best and only way to perform a mass transfer is to go over the internet. The only faster method is to ship disks, which is rarely even an option and is not widely supported by many cloud providers. Even with shipping disks it is likely that a significant chunk will still have to go over the wire. That being said, planning for a content migration to the cloud should start with identifying your internets upload speeds.There are plenty of ways to do this, such as If upload speeds are very slow you may need to consider buying dedicated lines so users are not affected by the bandwidth consumption. You may also want to consider only performing transfers at night or on the weekend. Unless you want to be the one babysitting transfer jobs over the nights and weekends, it’s a good idea to consider a tool to help you.

What content is being moved? More importantly, what can be left behind?

Hand in hand with how the speed of a connection is what and how much content will be moved. By understanding how many Gigabytes, Terabytes or Petabytes that need to be moved, an organization can better prepare for the demands it will place on its network and its staff. As the amount of data being moved to an EFSS platform increases, so do the complexities and the costs to the organization. In addition to pure data volume, the number and size of the files being moved can also impact how smoothly a migration runs. For example, sending very large files over the Internet can saturate an organization’s network uplink, triggering timeouts and transmission errors that can derail a migration or interfere with business critical network functions. Migrating large amounts of data or large numbers of files can also pose problems. This is because most EFSS systems employ rate limiting technologies that establish and enforce a maximum number of files and/or bytes per second. Unless it is managed well, this rate limiting function is often the primary cause of a data migration bottleneck, reducing the speed at which data can be moved onto the EFSS Platform.

Correctly scoping the content before it is transferred during the planning stage gives two important advantages when proceeding to execute the migration. The first is knowing how much time it will take based on your network capabilities. The second is equally as important – being able to make informed decisions about what can be left behind. Leaving data behind to be put into cold storage or deleted outright will save you countless hours and save the organization thousands of dollars in storage costs.

Security in motion is as important as security at rest

OK, now that you have an idea of what you will be moving and how long it might take, there are decisions that need to be made to ensure data is kept secure while it is in motion. Without proper planning, a data breach or loss would not only cause the migration process to fail but could be catastrophic for the business.

Read out post: Are Hidden Caches Putting Sensitive Data At Risk During A File Migration To The Cloud?

The two most important aspects of data security while in motion are encryption and ensuring data is never cached, persisted, or staged while in flight between the various systems. Using home made CLI tools that employ SSH or SFTP will most likely ensure both of these criteria are met, however, they have huge drawbacks such as loss of speed, lack of automation, and lots of troubleshooting. Normal FTP is incredibly insecure, storing passwords in plain text and not using encryption by default. User driven migrations using tools provided by the EFSS provider may be safe, however, there is likely to be data loss due to the unorganized nature of a user driven migration. In addition, no data is encrypted so if it ends up in the wrong place, it be seen by the wrong eyes. Third party tools may provide features for planning and some security advantages. But almost all third party tools’ architecture requires that data is copied at least once before reaching the target system. When using these migration tools you have no control over where those copies end up. Cloud FastPath is the only tool which does not persist any customer data – it is streamed directly between source and target.

Cloud FastPath architecture

To see the other fundamentals of a data migration as well as all of the migration options, download our ebook – Data Migration Fundamentals: What you need to know for simple, fast and secure cloud-based EFSS adoption.