1. Home
  2. Command Line Client
  3. CFP Command Line Client – Jobhistory commands

CFP Command Line Client – Jobhistory commands

The jobhistory command is one of the most widely used cfpclient commands. It can be used to:

  • access transfer reports from the command line
  • generate summaries of the bytes of transferred for billing purposes
  • view the errors and statistics associated with a given job

The best way to start a jobhistory inquiry on a particular job is:
python cfp jobhistory --list-jobs
This will generate a list of all jobs that have actually run at least once in your account. (Jobs that have not gotten past the validation phase will not be listed.):
+--------------------------+--------------------------+---------------+--------------------------+------------------------------------+
| first_start_time | last_end_time | num_instances | job_id | last_name |
+--------------------------+--------------------------+---------------+--------------------------+------------------------------------+
| Mon Jan 18 16:03:48 2016 | Mon Jan 25 16:52:32 2016 | 6 | 569d528dc1e9c66ae622be3e | win -> onedrive |
| Tue Jan 19 11:19:21 2016 | Wed Jan 27 12:51:18 2016 | 20 | 569e6193c1e9c66ae622e11c | google -> box |
| Wed Jan 20 13:25:00 2016 | Thu Jan 21 15:39:02 2016 | 4 | 569fcb5ec1e9c6510327d4a7 | win -> box |
| Wed Jan 20 13:26:55 2016 | Wed Jan 20 13:27:01 2016 | 1 | 569fd136c1e9c6510327d56c | box -> win |
| Wed Jan 20 13:38:45 2016 | Tue Feb 2 13:53:43 2016 | 4 | 569fd34cc1e9c6510327d5de | s3 -> dropbox |

Note that every job has an id listed next to the job name.  Choose the id for the job you want and enter the following to get a listing of all run instances for the job:
python cfp jobhistory --job 569fd34cc1e9c6510327d5de
+----------------+--------------------+----------------+--------------------------+-------------+----------+-----------+--------------------------+-----------------+---------+--------------------------+
| target_adapter | name | source_adapter | start_time | instance_id | source | state | end_time | account_mapping | target | job_id |
+----------------+--------------------+----------------+--------------------------+-------------+----------+-----------+--------------------------+-----------------+---------+--------------------------+
| dropbox | s3 -> dropbox 4636 | s3 | Wed Jan 20 13:38:45 2016 | 0 | amazonS3 | completed | Wed Jan 20 14:56:34 2016 | False | dropbox | 569fd34cc1e9c6510327d5de |
| dropbox2 | s3 -> dropbox 4636 | s3 | Wed Jan 27 10:45:23 2016 | 1 | amazonS3 | completed | Wed Jan 27 10:46:53 2016 | False | dropbox | 569fd34cc1e9c6510327d5de |
| dropbox2 | s3 -> dropbox 4636 | s3 | Fri Jan 29 16:04:51 2016 | 2 | amazonS3 | completed | Fri Jan 29 16:06:15 2016 | False | dropbox | 569fd34cc1e9c6510327d5de |
| dropbox2 | s3 -> dropbox 4636 | s3 | Tue Feb 2 13:52:17 2016 | 3 | amazonS3 | completed | Tue Feb 2 13:53:43 2016 | False | dropbox | 569fd34cc1e9c6510327d5de |
+----------------+--------------------+----------------+--------------------------+-------------+----------+-----------+--------------------------+-----------------+---------+--------------------------+

You can select the instance of the job you want and then extract data about them with these commands. The following command:
python cfp jobhistory --job 569fd34cc1e9c6510327d5de --instance 2 --log
reports out all of the events that were in the event log for this job instance.  Where the event log only holds the most recent 500 events, this is a way to recover events for a particular job.  This command:
python cfp jobhistory --job 569fd34cc1e9c6510327d5de --instance 2 --stats  --granularity second --end 2
reports out statistics and metrics for the job – items transmitted, number of errors, and so on – for the first two seconds of the job.  If you omit the –granularity and –end arguments from the above command, you get the default, which is the first hour of metrics.

To generate a transfer report:
python cfp jobhistory --job 569fd34cc1e9c6510327d5de --instance 2 --task-data transfer_report
To determine if a particular file transferred on any instances of a single job:
python cfp jobhistory --job 569fd34cc1e9c6510327d5de --task-data-history transfer_report --history-key filename --history-value "/folder/encoding\.py"

Jobhistory Billing Reports – for Total Data Transferred

To generate a report on how much data you have transferred in your account, you can enter:
python cfp jobhistory --billing

Updated on February 5, 2020

Related Articles