With Data Connector, you can automate data exports to DataCamp-owned S3. From here, with the support of your engineering team, it is then possible to connect these raw data files to your data warehouse or reporting platform of choice. Using Python or R, you can copy these files to your data lake or import them to your data warehouse and then connect these insights to your reporting platform of choice—whether that be Redshift, Snowflake, Tableau, PowerBI, Jupyter Notebooks, or other platforms.
- Configuring DataCamp Data Connector
- Example Python script to list and download S3 bucket content
- Explaining what the data represents
- Frequently Asked Questions
Configuring DataCamp Data Connector
- Navigate to your Reporting page in the left-hand menu.
- Select the Export tab at the top of the page and click “Set Up Data Connector”.
- This will create an S3 bucket on DataCamp’s AWS instance and display a modal with the auto-generated credentials you can use to access the S3 bucket containing your data exports. When you are ready click “Continue”. This will enable automated exports for your group.
- You can access your credentials at any time from the Reporting > Export page. You can also enable and disable the Data Connector through the toggle. Disabling the Data Connector will disable future data exports, but all other settings will remain. This will not delete your S3 bucket or erase the credentials you have configured, so you can use the same credentials and bucket if you choose to re-enable this feature.
- After completing the above steps, contact a data engineer in your organization to request they access your S3 Buckets using your credentials from Step 3. They can then assist you with connecting your data to your internal requirements, whether this is connecting to a data warehouse, like Redshift or Snowflake, or a reporting platform such as PowerBI, Looker, or Tableau.
- Once the Data Connector has been enabled via the Enterprise app, you will be able to access your learning data in the designated S3 bucket, in CSV format. You can use various tools to access and import the data from S3 to your local drive. We would suggest the following tools, both of which are very easy to use:
Below you will see the steps we followed to complete the import process using 3Hub:
- Click on the above link or search for 3Hub in the Apple Store.
- Download 3Hub to your laptop/computer.
- Once downloaded, go to DataCamp > Reporting > Export.
- Click on View Configuration Details on the Data Connector modal (assuming you have already enabled the Data Connector).
- Copy and paste the following configuration details from this modal to the 3Hub app interface: S3 bucket name, Access Key and Secret Key. (See the example below)
- Click 'Connect' after entering the necessary information.
- Once connected, you will be able to see all the data files in your designated S3 bucket.
- From this interface, you can save your daily learning data (or the latest learning data) to your local drive.
Note: This step can also be automated via a script, so you can write a Python script for example, which will automatically read the latest files from your S3 bucket and save it to your local drive, meaning you no longer need to manually download the files. You can find an example script below.
The example script requires only standard Python 3 installation and a Boto3 library which can be installed by executing the following command:
$ pip3 install boto3
The script has to be updated with AWS credentials and AWS S3 bucket name (lines 6,7,8 of the script) that can be found on the data connector modal (Click on View Configuration Details on the Data Connector modal).
List data connector S3 bucket content
When executed without arguments, the script will list S3 bucket content.
$ python3 data_connector_s3.py
Download the latest export from the S3 bucket
When executed with the argument: “latest” the script will download the content of the latest export from the S3 bucket into the local directory.
$ python3 data_connector_s3.py latest
Download specific export from the S3 bucket
When executed with an argument which is the date of the export, the script will download the content of the specified export from the S3 bucket into the local directory.
$ python3 data_connector_s3.py 2021-08-14
Example Python script to list and download S3 bucket content
An example of the simplest Python script which uses the popular AWS Python library Boto3 to access S3 content is provided below. Note that this is an example that can and should be adjusted to your organization’s specific workflow.
aws_access_key = "ACCESS_KEY"
aws_secret_key = "SECRET_KEY"
aws_bucket = "BUCKET_NAME"
s3 = boto3.resource('s3', aws_access_key_id=aws_access_key, aws_secret_access_key=aws_secret_key)
file_list = 
bucket = s3.Bucket(bucket_name)
for bucket_object in bucket.objects.all():
def download_from_s3(dirname, bucket_name=aws_bucket):
bucket = s3.Bucket(bucket_name)
current_directory = os.getcwd()
target_directory = os.path.join(current_directory, dirname)
for object in bucket.objects.filter(Prefix = dirname):
if len(sys.argv) == 1:
elif sys.argv == 'latest':
current_date = datetime.datetime.today().strftime('%Y-%m-%d')
Explaining what the data represents
We have created a short guide explaining the Data Model so you can understand what information is available to track user progress on a daily basis. As the automated export is a raw, unfiltered export of your organization’s learning data, you will be required to join the appropriate tables, to extract the metrics you care the most about. The following document will guide you through the content and dimension of each table and how you can best join them together: Data Connector: What data is accessible for analysis?
Frequently Asked Questions
Can I directly integrate the Data Connector with my BI tool, like PowerBI or Tableau?
At the moment, we do not support that integration, but we plan to directly support the JDBC driver via AWS Athena to access S3 content in the future.
Who has access to the DataCamp Data Connector?
The data export tab and the DataCamp Data Connector can only be accessed by group admins.
What is the DataCamp Data Connector and is it a fit for us?
DataCamp Data Connector allows customers with our business plan to access their raw learning data, including some additional data that is not easily available in the Enterprise app. By enabling this functionality, you will no longer need to manually log into the DataCamp platform and manually export the learning data. This feature is fit for companies with robust analytics and business intelligence requirements.
Which kinds of companies are successful with the DataCamp Data Connector?
This will be most beneficial to companies who manually export their learning data and spend a significant time manipulating it to get different views (metrics over time, department level, etc.). These companies have a good understanding of the KPIs and learning progress data they want to see.
What can I achieve with the DataCamp Data Connector?
- Understand the impact of learning and development efforts, communicate progress, diagnose bottlenecks, drive decision-making, and predict development needs.
- Access most of your raw learning progress data, including some additional data that is not currently available in the Enterprise app.
- Create a variety of visualizations: trends over time, pie charts, bar graphs, etc.
- Slice and dice data with a variety of filters, for example, department, office, recruiter, etc.
What are some of the limitations of the BI Connector?
- Not for activity tracking or tracking hourly data; not for instant analytics—data is refreshed every 24 hours.
- The DataCamp Data Connector is meant to have dedicated users who create reports for the whole team and preferably are Data Analysts.
How often is the data updated?
The data is synced to Amazon S3 daily (including weekends). Your organization can connect your in-house BI tool to Amazon S3 to pull your data from DataCamp.
Can I access the files through my own S3 bucket?
Currently, we only support data transfer via Datacamp Amazon S3. We will provide you with secure credentials to access your learning data.
Are my credentials stored securely?
Your secret password is encrypted and stored securely on AWS SSM params store. DataCamp does not store this information in any database. It is only displayed upon request. All admins within your group will see the same credentials.
Can different admins create their own set of AWS credentials?
No, the credentials are created based on the group ID. Therefore, all admins will have access to the same credentials regardless of who sets up the initial configuration.
Is the data in the DataCamp Data Connector backed up?
We do not create backups of the DataCamp Data Connector.
How do I enable the DataCamp Data Connector?
You can enable the DataCamp Data Connector either via the Enterprise app or through your Customer Success Manager, if you have an Enterprise, Usage, or Unlimited DataCamp.