Job History On-screen Transfer Log
The Job History On-screen Transfer Log provides an on-screen listing of the files transferred in a given run of a job. You also have an option to export the transfer log to a file for permanent documentation of your migration.
While CFP job reporting provides a rich and extensive bank of the files/folders, metadata, ownership and sharing in your jobs. For a job of any size, this can result in reports that, while they can be exported and downloaded, they are too big to open in Excel. Fortunately, there are many ways to filter reports to reduce the .xlsx file size so you have the information you specifically need to make decisions and document your migration.
IMPORTANT: Always review the accounting line of the event log in conjunction with reviewing the Transfer Log. See below for details.
Accessing the On-screen Transfer Log
- From the CFP dashboard, click on the job name, then select the History button.
- Select the View button for the job run that you wish to look at. Select Export to download a .csv file of your transfer.
Filtering the On-screen Transfer Log: Statuses and Errors
1: Find the ‘All Statuses’ menu and click on it.
2: The menu will list all file statuses that have been logged to date for all files in the run. Please note, status and error messages are not the same as status messages are informational. Some of the statuses in this menu may include:
- read-errors: due to issues such as permissions, file open/EBUSY or source connectivity, typically on the source side
- write-errors: due to issues “writing” the data to the target, such as file exceeding target account limits
- filtered: files that are not transferred to the target. These include system files, such as desktop.ini and thumbs.db files, or temp files such as ~$doc.docx. In many cases, these files may cause undesirable results if downloaded onto another computer from the cloud provider, or they may be prohibited altogether by the cloud provider. Files will also be filtered if there is an explicit exclude filter set up for them in Advanced Options, or if they are skipped in the spreadsheet of an account mapping job.
- match-exists: an up-to-date copy of the file is already on the target, so the file was not transferred
- success: file transferred successfully
- partial-success: the file or folder transferred successfully and completely, but some of the associated metadata did not transfer. A common source of this error is if a folder transfers, but one or more of the associated sharees did not transfer. For example, if you are migrating a folder to Dropbox and one of the designated sharees has not confirmed their invitation to Dropbox, they cannot be attached as a sharee to that folder.
- source-missing: the vast majority of source-missing files are ones that were simply found on the target but not on the source. These are informational and do not imply there is a problem. Source-missing files that have a Message of “Not found” are generally files that have been deleted from the source while the job was in progress, and if so, the files and associated errors will not be reported on subsequent runs unless the file is re-added to the source. More details available further in this article.
- size-exceeded: file is larger than the target service supports.
- target-account-missing: the affected file or folder had a source owner that was not mapped to a target account. The affected item did not transfer because you did not specify where you wanted it to go on the target. This error is only present in account mapping jobs.
- match-future-time: if the source mtime is in the future, CFP will send that mtime to the target, but most targets will convert that to the time the file is transferred, because they don’t support future mtimes. That sets up a scenario where the file will always be newer on the source than target. To prevent re-sending the file unnecessarily on syncs, CFP checks all other file parameters, such as ctime and file size, and if they match, it will not transfer the file and will report it as ‘match-future-time.’ For further information on how files may have mtimes in the future, see https://en.wikipedia.org/wiki/Year_2038_problem.
3: Select the file status filter of interest to display just those files. Refer to Job History Filters for additional information on selecting data of interest from your job.
In most cases, CFP is simply reporting errors it receives from the source or target. For read-errors from Windows sources in particular, that means you can frequently just do a web search for the errors and find Microsoft documentation that explains the issue or, if you have access to the server, you can try to directly access the affected file for folder and open or download it. You will frequently get the same error.
To identify the specific error for a given item, simply select the item in the Files Transferred tab of job history and then look for the error message in the Details pane for that file/folder at the bottom of the page.
The errors that you can encounter in a job are numerous, and change over time as cloud service providers update their services. Moreover, the error received may have different causes depending on the context in which it occurs. If you don’t understand why you received an error and it doesn’t resolve on re-run of the job, contact firstname.lastname@example.org and they will be happy to assist.
Exported Transfer Log
The Export button at the bottom of the Files Transferred tab provides you with an xlsx copy of the transfer. This has two advantages: you can send the report to someone else who does not have access to CFP, and you have permanent documentation of your transfer after your CFP license has expired. Additionally, the Transfer Log has a Translated File Name column at the far right, which provides information on the precise path to every file in your job.
The All Results menu at the top right of the Files Transferred pane is multi-select, so that you can export several file statuses for your job in a single file. Simply open the menu and select all of the desired statuses that you want:
Additionally, there is an extensive list of filters for the source, target and job data in general by clicking on the magnifying glass icon next to the All Results menu. These can be used in conjunction with the All Results menu to filter down the reports.
Last, you can also filter the exported report by tab. When you click the Export button, a pop-up window will enable you to decide which tabs of the report to include:
Note that you can deselect sub categories of Users, Collaborations, and Collaborators so that only source or target data is included in the report. In the report above, the Transfer Overview and Target Collaborators have been deselected.
Transfer Report Tabs and Their Content
Transfer Overview: Provides a summary of statistics for the job.
Files & Folders: A list of all files and folders in the job along with a status for each one on what happened to it in the job. Includes metadata including file/folder id’s, last modified time, source/target owner, full target paths, and, if configured, last modifier.
Users: A list of all selected users in the source node for source users, and all target users that have received that data in the target users tab. This tab enables you to quickly understand which source accounts own large stores of data, and conversely, source accounts that may be activated, but own no data. It will tell you target accounts that are approaching storage limits.
Collaborators: A list of all users who are sharees of data in the job. Again, source and target sharees are listed in separate tabs.
NOTE: If you click on a collaborator name in the Collaborators tab of the UI, you will get a filtered list of all of the files and folders that are shared with that collaborator.
Collaborations: A listing of all paths that are shared. A path will be listed with one collaborator per line, so if a path has multiple sharees, it will be listed multiple times. Links used to share data are listed in the collaborators tab for some services, including Box and GDrive, but are not applied during the job. Only ACL shares are applied, but the report does enable users to manually apply link shares to the target or otherwise ensure that those sharing relationships are effected.
The Collaborations tab is particularly useful for determining which data should go to Team environments – such as Sharepoint in Microsoft or Team Folders in Dropbox – and which data should go to more personal spaces such as OneDrive in Microsoft or personal accounts in Dropbox. In general, highly shared data is often best suited to Team environments and unshared data is more suited to personal accounts. However, CFP fully supports shared data in both environments.
Event Log: A list of all events that occurred during the job. These can be helpful for post migration analysis.
Most source-missing files are files that are found on the target, but not the source. In most cases, these files are listed for informational purposes only – CFP is just telling you there are additional files on the target that are not on the source. However, there is an unusual occurrence when source-missing files may be listed as read-errors in the event log, and they are not necessarily present on the target.
If a file is found on the source during scanning, but is deleted before the file is ready to be transferred, it will generate an error of not_found or Item is trashed. This will be reported to the event log as a read error, but the file will be listed in the transfer report as source-missing – even if the file is not on the target.
In this case, the file was presumably no longer wanted by the end user – so we list it as source-missing rather than a read-error in the transfer report. If the job re-runs, because the file is no longer present, the file and its associated error will not be reported.
Sending Transfer Log to Target
You can also have a copy of the transfer report sent to the target for any job instance that ends in complete or incomplete. Simply check off ‘Write report file’ in the job settings before running the job. The default for this checkbox is deselected; note that transfer reports sent to the target will count towards billable bytes transferred and the total number of files transferred.
Reviewing the Event Log
In rare instances, you may find directory access errors that are listed in the event log but not in the transfer report. To find out if you have directory access errors, review the accounting line near the top of the event log. It will look like this:
5060 files transferred, 0 already in sync, 2077 directories created, 2 skipped files due to include/exclude filters.
If you have any directory access errors listed, review the event log to determine the names of the directories. You can filter the event log for errors from the “All Severities” menu in the upper right corner of the log.