Amazon S3, DropBox, Google Drive, and OneDrive CSV Configuration
We can produce a Comma Separated File (CSV) and read them from Amazon S3, DropBox, Google Drive, and OneDrive. This is a simple way to import data from other services or to export files for backups or sharing with other applications. To allow the maximum flexibility and support with all services, the configuration is quite rich. Here is how you configure it:
- Set the Path to the file you wish for us to read or modify
- Indicate the separator we should use for the file. If you do not find the one you need, just contact us.
- If your file contains comment blocks, select the comment prefix character. If you do not find the one you need, just contact us.
- If your file contains a block of text at the top, you can change the starting row to the row number where the headers are located.
- For each column in your data, select the checkbox to the left of the Column Number and then enter the numeric index of the column. For example, in Excel, if the column is A2, this value would be 2.
- For any data which you do not have in your data, you can enter an override value in the Default column.
- Some CSV files are daily summaries of activity data, and in such cases, you may wish to group these by the date so that you do not have a lot of records per day (such as one per minute). To do so, check the “Group data by Date?” checkbox (this is presently only available for Activities).
- Some CSV files have the verbose data per meal, and in such cases, you may wish to group these by the date so that you do not have a lot of small records. To do so, check the “Group data by Meal?” checkbox (this is presently only available for Nutrition Information).
- Some CSV files come with non-qualified time zone, and in such cases, we default to your Profile’s time zone. If the unqualified date and time is actually in UTC, you can override it via the “Force UTC time zone for non-qualified dates?” checkbox.
To help with the column mapping, you can select the “Choose Columns by Uploading Sample” button which will allow you to upload a sample portion of the file, then interactively map the columns and save that configuration. This is a great way to ensure that you have all of the required fields, such as a start date.
All other settings are just like the Sources or Destinations.
Some services may produce CSV files which represent an activity. Such files are supported, however, you must use the Map-based option to configure this.
To configure Amazon S3, place your Access Key into the username field, your Secret Key into the Password field, and for the path, use bucket/directory/file.
If you receive an error indicating that a row does not have a date provided or an activity or description, more than likely you are missing a required field or that it was unable to be parsed. We recommend using our sample uploader which will list the required fields for that data type.
Note: For destination tasks, we do not rewrite the file but rather append the file with what we believe is the new information. If you delete the file and want to repopulate it, you will need to Reset the task.