Multipart Upload

Requesting a multipartUploadTicket returns all information necessary to perform a file upload directly to S3.

Files uploaded using this process do not have a limited file size.

Attribute	Description
fileName	This is the file name as it will be stored in S3. It does not have to be the actual name of the local file.
sessionId	A session id is passed in to refresh existing, unexpired temporary credentials. Temporary credentials expire after 1 hour. To complete an upload that takes longer than 1 hour, temporary credentials must be refreshed. Passing in the sessionId creates new credentials that have access to that specific upload session allowing you to continue the upload. Refreshing credentials updates the expiration to be 1 hour from the time they were refreshed.

Attribute

Description

fileName

This is the file name as it will be stored in S3. It does not have to be the actual name of the local file.

sessionId

A session id is passed in to refresh existing, unexpired temporary credentials. Temporary credentials expire after 1 hour. To complete an upload that takes longer than 1 hour, temporary credentials must be refreshed. Passing in the sessionId creates new credentials that have access to that specific upload session allowing you to continue the upload. Refreshing credentials updates the expiration to be 1 hour from the time they were refreshed.

Overview of Multipart Upload

Step 1: Create a multipart upload ticket. This will have all the information needed to upload the asset to S3. This includes fileName, sessionId, assetUrl, expiration, accessKey, secretKey, sessionToken, objectKey(path in S3 where the file is stored), and bucketName.

Refreshing Temporary Credentials: (Temporary Credentials expire after 1 hour from the time they were created or refreshed.)
When requesting temporary credentials, you pass in the file name. When you start uploading the file to S3, the sessionId is the reference to that upload session. To refresh temporary credentials, we pass the file name and the sessionId. This way, the refreshed credentials have access to that specific upload session. Then the credentials are interchangeable since they have the same access.
There is a code example on refreshing credentials using the Mediasilo API further down this page.

Step 2: Upload the asset to S3. Once the upload is complete, the assetUrl will be accessible to create the asset in your Mediasilo account. NOTE** Although the assetUrl is returned in the multipart upload ticket, the assetUrl is not accessible until the upload is complete.

Step 3: Create the asset. Using the assetUrl and projectId, create the asset inside the desired project. Note** the projectId is the id of an EXISTING project inside of your Mediasilo account.

Step 1: Create the Multipart Upload Ticket

Let's assume we sent the following request to obtain a multipart upload ticket:

POST /v3/assets/multipart/upload
{
    "fileName": "FILE0013.MP4"
}

And we receive the following payload in response:

//200 OK
{
    "fileName": "FILE0013.MP4",
    "assetUrl": "https://s3.amazonaws.com/s3-bucket/9XXXXXX-7XX9-4XX5-bXX2-b07XXXXXX40b/FILE0013.MP4",
    "expiration": 1507790546000,
    "accessKey": "ASIAJCPLGOLJYYTKUXRQ",
    "secretKey": "TWfwinpU3YRUXzIN/UoSCqsoobOYKBAIpWMC4Si3",
    "sessionId": "9XXXXXX-7XX9-4XX5-bXX2-b07XXXXXX40b",
    "sessionToken": "FQoDYXdzEOf//////////wEaDDwBJ3jkRXXXXXXXXXXXXXXXXXXXXXXXvu8hRdXhhb9KEV3VUTiKxHJGmgkT3nd2FbNRJrWYLVG7acZk+Qr4XXXXXXXXXXXXXXXXXXXXG2g6K2tlh/QgMBGLBQ+/nMli2cKuh/CKGSZ3KibspxO+0t35v5rXXXXXXXXXXXXXXXXXXXXXXpYyhih7u1Q8iRSxmtjMf9uRg7RIZa8usnPRZhuAIU9Kc7Y+pJ8gIjTRZQdx5bdaU6hPJQXXXXXXXXXXXXXXXR9XK8Io4m49KzeXV6vKgAd9OV77f0kUc3mOVK2PMrzvOWCTykAjvYeC9XXXXXXXXXXXXXXXXXdEvuiDPZoMAPZ8cXRy3ts5WEr8eVBIaEj+nEtFQrAww3NCytRSXXXXXXXXXX208pDwsVNSMeKMKB/M4F",
    "objectKey": "9XXXXXX-7XX9-4XX5-bXX2-b07XXXXXX40b/FILE0013.MP4",
    "bucketName": "s3-bucket"
}

Attribute	Description
fileName	The asset's file name.
sessionId	This id refers to the aws S3 upload session of your asset.
assetUrl	This is the url that points to the asset that has been uploaded to S3
expiration	The exact time your temporary credentials expire in milliseconds.
accessKey	Temporary AWS access key.
secretKey	Temporary AWS secret key.
sessionToken	AWS session token.
objectKey	This is the path where your asset will be uploaded to in S3.
bucketName	Name of the bucket in S3 where your asset will be stored.

This response contains all we need to start the upload process. It also includes the assetUrl which will be used at the end when we create the asset.

Step 2: Multipart Upload

Multipart Upload can be done in a browser (via JavaScript) or using any number of languages (Java, PHP, Python, Node, etc.). We take advantage of the AWS Javascript SDK to accomplish this. This SDK is supported in many languages as well.

For supported file types, take a look at our support documents.

var credentials = new AWS.Credentials({ 
  accessKeyId, 
  secretAccessKey, 
  sessionToken 
});
var service = new AWS.S3({ credentials });
var managedUpload = new AWS.S3.ManagedUpload({ 
  params: { 
    Bucket, 
    Key, 
    Body 
  }, 
  service 
});
managedUpload.computeChecksums = true;// must be true. see docs for more details
managedUpload.send();

The multipartUploadTicketObject allows us to create a credentials object. (Amazon Docs) http://docs.aws.amazon.com/AWSJavaScriptSDK/latest/AWS/Credentials.html
The credentials object allows us to create an S3 service object. (Amazon Docs) http://docs.aws.amazon.com/AWSJavaScriptSDK/latest/AWS/S3.html
The S3 service object allows us to create a ManagedUpload object. (Amazon Docs) http://docs.aws.amazon.com/AWSJavaScriptSDK/latest/AWS/S3/ManagedUpload.html

The managedUpload.send() call will start and finish the multipart upload.

📘
Data Fields
All data needs to be passed along exactly as it was returned in the ticket request.

3. Create Asset

Once the file is uploaded successfully you have 24 hours to create an asset before the content is deleted. create the asset using the API as usual.

POST /v3/assets
{
    "sourceUrl": "https://s3.amazonaws.com/s3-bucket/26a35865-282f-4aa2-b9af-1999d2c7835b/myfile.mov",
    "projectId": "0XXXCC-014B-2XX0-CF518DXXXX393E"
}

4. Refreshing Credentials

This defines the refresh function for the AWS.Credentials object. It can now be used on any Credentials object in scope.

AWS.Credentials.prototype.refresh = function (callback) {
            var self = this;
            refreshTemporaryCredentials(function (err, response) {
                if (err) {
                    callback(err);
                } else {
                    self.accessKeyId = response.accessKey;
                    self.secretAccessKey = response.secretKey;
                    self.sessionToken = response.sessionToken;
                    callback();
                }
            });
        };

This is the function that makes a POST request to the Mediasilo API to get the refreshed credentials. Remember that refreshed credentials contain a new access and secret key but retains the same sessionId. New temporary credentials with access to the same upload session.

function refreshTemporaryCredentials(callback) {
    var xhrRefreshCreds = new XMLHttpRequest();
    xhrRefreshCreds.open('POST', 'https://api.mediasilo.com/v3/assets/multipart/upload', true);
    xhrRefreshCreds.setRequestHeader('Authorization', 'Basic ' + btoa(userName + ":" + password));
    xhrRefreshCreds.setRequestHeader('Content-Type', 'application/json');
    xhrRefreshCreds.setRequestHeader('MediaSiloHostContext', hostName);
    xhrRefreshCreds.onreadystatechange = function () {
        if (xhrRefreshCreds.readyState == 4 && xhrRefreshCreds.status == 200) {
            console.log("Finished refreshing temporary credentials");
            var refreshedCreds = JSON.parse(xhrRefreshCreds.responseText);
            callback(null, refreshedCreds);
        }
     }
            xhrRefreshCreds.send(JSON.stringify({ 'fileName': fileName,         'sessionId': sessionId }));
}

This is the refresh interval that determines when to call the refresh function. We are refreshing every 30 minutes to ensure that no AWS requests are made with expired credentials.

var refreshInterval = setInterval(() => {
   credentials.refresh(function () {
       console.log("Temporary Credentials have been successfully refreshed.")
   });
}, 1800000);

Here is a full JavaScript example:

If you have an existing Mediasilo account, you will be able to run this example fairly easy.

Copy and Paste this code into a file, and name it "index.html".
Because you already have a Mediasilo Account, you can fill in the variables hostName, userName,
and password, with the same values that you use to login with.
Get the project Id from any existing project in your account, and set that as your projectId:
Follow steps a - d to get a project id if you don't know how:
a. Login into your Mediasilo Account. Make sure you have at least 1 project created.
b. Open the browser's Dev Tools (I recommend using Google Chrome: hotkey = cmd+option+i).
c. Now that you have the Dev Tools open, refresh the page.
d. Click the "Network" tab, and you can now filter through the requests using the keyword
"project". The response to that request will have your Id.
Once you have hardcoded hostName, userName, password, and projectId, you can open your index.html file in a browser, and upload a file.

IMPORTANT NOTE
This example is for teaching purposes. You should never hardcode sensitive data like passwords in live applications.

<!DOCTYPE html>
<html>

<head>
  <meta charset="UTF-8">
  <title>MediaSilo API - JS Multipart Upload Example</title>

  <!--
    The script tag below gives us access to the AWS Javascript SDK.
    This allows us to use the MangedUpload class, which handles 
    the entire multipart upload process.
    -->
  <script src="https://sdk.amazonaws.com/js/aws-sdk-2.20.0.min.js"></script>

  <script>

    /*
    *Variables that you have to fill in.
    *NOTE* You must have an active Mediasilo account. The same hostName,
    *userName, and password you use to login to mediasilo.com with, are used here.
    */
    var hostName = '';
    var userName = '';
    var password = '';
    var projectId = '';//This id references a real project in your Mediasilo account. This will be the final destination for the new asset.

    /*
    *Global variables needed throughout the asset create process.
    */
    var fileName;//The asset's file name.
    var sessionId;//This id refers to the aws S3 upload session of your asset.


    /*
    *When the upload button in the browser is clicked, this function 
    *will start the asset create process.
    *
    *Step 1: Create a multipart upload ticket. This will have all the information 
    *needed to upload the asset to S3. This includes fileName, sessionId, assetUrl, expiration, 
    *accessKey, secretKey, sessionToken, objectKey(path in S3 where the file is stored), and
    *bucketName.
    *
    *Step 2: Upload the asset to S3. Once the upload is complete, the assetUrl will be  
    *accessible to create the asset in your Mediasilo account. NOTE** Although the assetUrl
    *is returned in the multipart upload ticket, the assetUrl is not accessible until the
    *upload is complete.
    *
    *Step 3: Create the asset. Using the assetUrl and projectId, create the asset inside the
    *desired project. Note** the projectId is the id of an existinbg project inside of your
    *Mediasilo account.
    */
    function startAssetCreateProcess() {
      //Step 1:
      createMultipartUploadTicket();
    }

    function createMultipartUploadTicket() {
      var filePath = document.getElementById('file').value;//The exact path to the file on your computer.
      fileName = filePath.replace(/^.*?([^\\\/]*)$/, '$1');//The name of your file which is extracted from the file path.

      /*
      *This is the POST request that is made to the API that returns the              
      *information needed to uploada file to S3, then later create an asset 
      */
      var xhrMultipartUploadTicket = new XMLHttpRequest();
      xhrMultipartUploadTicket.open('POST', 'https://api.mediasilo.com/v3/assets/multipart/upload', true);
      xhrMultipartUploadTicket.setRequestHeader('Authorization', 'Basic ' + btoa(userName + ":" + password));
      xhrMultipartUploadTicket.setRequestHeader('Content-Type', 'application/json');
      xhrMultipartUploadTicket.setRequestHeader('MediaSiloHostContext', hostName);
      xhrMultipartUploadTicket.onreadystatechange = function () {
        if (xhrMultipartUploadTicket.readyState == 4 && xhrMultipartUploadTicket.status == 200) {
          console.log("Finished creating the multipart upload ticket.");
          //Step 2:
          uploadFileToS3(xhrMultipartUploadTicket.responseText);
        }
      }
      xhrMultipartUploadTicket.send(JSON.stringify({ 'fileName': fileName }));
    }

    function uploadFileToS3(multipartUploadTicket) {
      var Body = document.getElementById('file').files[0];
      var multipartUploadTicketObject = JSON.parse(multipartUploadTicket);

      //This is the multipartUploadTicketObject deconstructed so we can clearly see the information we have access to.
      fileName = multipartUploadTicketObject.fileName;
      sessionId = multipartUploadTicketObject.sessionId;
      var assetUrl = multipartUploadTicketObject.assetUrl;//This is the url that points to the asset that has been uploaded to S3
      var expireTime = multipartUploadTicketObject.expiration;//The exact time your temporary credentials expire.
      var accessKeyId = multipartUploadTicketObject.accessKey;//AWS access key.
      var secretAccessKey = multipartUploadTicketObject.secretKey;//AWS secret key.
      var sessionToken = multipartUploadTicketObject.sessionToken;//AWS session token.
      var Key = multipartUploadTicketObject.objectKey;//This is the path where your asset will be uploaded to in S3.
      var Bucket = multipartUploadTicketObject.bucketName;//Name of the bucket in S3 where your asset will be stored.

      var credentials = new AWS.Credentials({ accessKeyId, secretAccessKey, sessionToken });//The multipartUploadTicketObject allows us to create a credentials object
      var service = new AWS.S3({ credentials });//The credentials object allows us to create an S3 service object.
      var managedUpload = new AWS.S3.ManagedUpload({ params: { Bucket, Key, Body }, service });//The S3 service object allows us to create a ManagedUpload object.
      managedUpload.computeChecksums = true;
      managedUpload.send();
      managedUpload.promise().then(function () {
        //Step 3:
        createAsset(assetUrl);//We do not want to create the asset until the upload to S3 in finished.
      });
      
      //This will update the upload progress bar everytime the httpUploadProgress event is omitted.
            //The event is omitted every time a single part is uploaded successfully
            managedUpload.on('httpUploadProgress', function (progress) {
                document.getElementById("progressNumber").value = (progress.loaded / progress.total) * 100;
                //console.log((progress.loaded / progress.total) * 100);
            })

      /*
      *Temporary Credentials expire 1 hour after they are created or refreshed. 
      *This interval will refresh the temporary credentials every 55 minutes.
      *This ensures that uploads needing longer than one hour will complete.
      */
      var refreshInterval = setInterval(() => {
        credentials.refresh(function () {
          console.log("Temporary Credentials have been successfully refreshed.")
        });
      }, 1800000);//1,800,000 milliseconds = 30 minutes. Reducing this number will increase the frequency at which your temporary credentials are refreshed.
    }//If this number exceeds 3,599,999 then your upload will fail before your temporary credentials are refreshed. The slower your internet
    //upload speed, the more frequent you want to refresh your credentials.

    function createAsset(assetPath) {
      /*
      *This is the POST request that is made to the API that creates your asset.
      *This will return the id of the asset that was created. 
      */
      xhrCreate = new XMLHttpRequest();
      xhrCreate.open('POST', 'https://api.mediasilo.com/v3/assets', true);
      xhrCreate.setRequestHeader('Authorization', 'Basic ' + btoa(userName + ":" + password));
      xhrCreate.setRequestHeader('Content-Type', 'application/json');
      xhrCreate.setRequestHeader('MediaSiloHostContext', hostName);
      xhrCreate.onreadystatechange = function () {
        if (xhrCreate.readyState == 4 && xhrCreate.status == 200) {
          //All Done! Your asset has been created.
          alert('Asset Created! Response: ' + xhrCreate.responseText);
        }
      }
      xhrCreate.send(JSON.stringify({ 'sourceUrl': assetPath, 'projectId': projectId }));
    }

    /*
    *This block of code defines how the refresh() function will work.
    *Any AWS.Credentials object that is created may use this refresh function.
    */
    AWS.Credentials.prototype.refresh = function (callback) {
      var self = this;
      refreshTemporaryCredentials(function (err, response) {
        if (err) {
          callback(err);
        } else {
          self.accessKeyId = response.accessKey;
          self.secretAccessKey = response.secretKey;
          self.sessionToken = response.sessionToken;
          callback();
        }
      });
    };

    /*
    *This function is almost identical to the createMultipartUploadTicket. The main difference
    *is that we are passing along a sessionId with a fileName when we make the POST request. 
    *With the addition of the sessonId, the will API know to refresh your credentials.
    * 
    *NOTE**This refresh function does actually return new temporary credentials. The access key
    *and secret key will be different than the previous keys. The 2 important things to understand
    *are that 1: the new temporary credentials have the exact same permissions as the previous credentials,
    *and 2: the expireTime on the new temporary credentials is roughly 1 more hour than the previous
    *credentials. This allows the credentials to be interchangeable, even in the middle of an upload. 
    */
    function refreshTemporaryCredentials(callback) {
      var xhrRefreshCreds = new XMLHttpRequest();
      xhrRefreshCreds.open('POST', 'https://api.mediasilo.com/v3/assets/multipart/upload', true);
      xhrRefreshCreds.setRequestHeader('Authorization', 'Basic ' + btoa(userName + ":" + password));
      xhrRefreshCreds.setRequestHeader('Content-Type', 'application/json');
      xhrRefreshCreds.setRequestHeader('MediaSiloHostContext', hostName);
      xhrRefreshCreds.onreadystatechange = function () {
        if (xhrRefreshCreds.readyState == 4 && xhrRefreshCreds.status == 200) {
          console.log("Finished refreshing temporary credentials");
          var refreshedCreds = JSON.parse(xhrRefreshCreds.responseText);
          callback(null, refreshedCreds);
        }
      }
      xhrRefreshCreds.send(JSON.stringify({ 'fileName': fileName, 'sessionId': sessionId }));
    }

  </script>

</head>

<body>
  <span>File</span>
  <input type="file" id="file" name="file" size="10" />
  <input id="uploadbutton" type="button" value="Upload" onclick="startAssetCreateProcess();" />
    
    <p>Upload Progress
        <meter id="progressNumber" min="0" max="100" value="0">350 degrees</meter>.
    </p>
</body>

</html>

Overview of Multipart Upload

Step 1: Create the Multipart Upload Ticket

Step 2: Multipart Upload

📘Data Fields

3. Create Asset

4. Refreshing Credentials

Here is a full JavaScript example:

📘
Data Fields