How to process files from Amazon S3 using GCS bucket

MapTiler Cluster is now available only on the Google Cloud Platform, using Google Cloud Storage buckets as input and output for the processing. However, it is still possible to easily transfer your data into Google Cloud Storage from the Amazon S3 bucket, from a list of object URLs, or another Google Cloud Storage bucket.

Processing files from Amazon S3

Transfer data from Amazon S3 bucket

Google Cloud Storage supports the transfer of data from the Amazon S3 bucket. Go to Google Cloud Storage Console, where you would set up the data transfer. Click on the CREATE TRANSFER JOB button.

First, you need to select the service from which the data would be transferred. In this case, you are going to transfer data from Amazon S3 bucket. In the Amazon S3 bucket field type name of the S3 bucket, the name is the same as it appears in the AWS Management Console. Then provide Access key ID and Secret access key. For more information visit official documentation.

In the next step, you have to provide the destination bucket. Insert the name of the bucket into the Cloud Storage bucket field and click Continue. 

If you want to create a new bucket, click on the Browse button on the right side, then click on the icon with a bucket and the plus sign as NEW BUCKET, and provide the unique name of the bucket and click Create.

In the last step, you configure the transfer, depending on your use case you can run transfer once by choosing option Run now or run it daily by choosing Run daily at where you specify the time when it should run. In the Description provide the unique name of the transfer or use the auto-generated name for the transfer. Then click Create button for transfer creation.

Now you can start using the transferred data in the MapTiler Cluster.

For further information about Google Cloud Storage transfer process visit official documentation for transfer.

Transfer sample data from Google Cloud Storage bucket

By following step-by-step this tutorial, you will transfer our sample data of San Diego aerial photos from our public mtc-sample-data Cloud Storage bucket into your Google project Cloud Storage bucket.

Start at Google Cloud Storage Console by setting up the data transfer.

First, you need to select the service from which the data would be transferred. In this case, you are going to transfer the data from Google Cloud Storage bucket source. In the Cloud Storage bucket field type mtc-sample-data which is our public sample dataset.

In the next step, you have to provide the destination bucket. Insert the name of the bucket into the Cloud Storage bucket field and click Continue. 

If you want to create a new bucket, click on Browse button on the right side, then click on the icon with a bucket and plus sign (see below) as a NEW BUCKET, and provide the unique name of the bucket and click Create.

In the last step, configure the transfer, in this case, you can leave the checked option Run now. In the Description field, provide the unique name of the transfer or use the auto-generated name for the transfer.

After you click on the Create button, you will see the status of the transfer process.

After the transfer process is finished, use the transferred dataset for test rendering with MapTiler Cluster.

Useful links

Official documentation for transfer
MapTiler Cluster