Allowing external users to securely and directly upload files to Amazon S3

Organizations are often required to store files, images, and other digital assets in a repository. In many cases, the source of these files are partners or individuals who are not connected to internal systems and requires corporate authentication in order to upload the files. Customers traditionally use servers to handle file uploads, which can use a significant amount of network I/O and computing resources. In addition, it can be extremely challenging to keep the file uploads performant as the application scales. To meet these requirements, organizations frequently maintain costly and time-consuming infrastructure to store and distribute digital assets, which can often be less secure and more expensive than alternative setups.

In this post, we demonstrate how customers can build a modern web application to securely upload multiple files directly to Amazon Simple Storage Service (Amazon S3)  using AWS Amplify. This solution provides a secure method of allowing external users to upload files to Amazon S3 without providing them direct access to your storage resources.  In addition, it allows you to leverage a cost-efficient and fully serverless architecture that reduces the operational burden of managing servers while also taking advantage of the scalability and reliability of Amazon S3.

Solution overview

AWS Amplify is a set of purpose-built tools and features that lets frontend web and mobile developers quickly and easily build full-stack applications on AWS, with the flexibility to leverage the breadth of AWS services as their use cases evolve. The service makes it simple for anyone to build web and mobile applications without having to manage the underlying infrastructure. The AWS Amplify framework provides libraries for storage, authentication, GraphQL, and many more.

In this solution, we use Amazon CloudFront for content delivery and we authenticate users of the web application from the browser through Amazon Cognito, allowing them to upload files to our S3 bucket, which is the storage component. To simplify the deployment of this solution, we use AWS Cloud9.

Figure 1: Using AWS Amplify to upload files from a browser to Amazon S3

The workflow and architecture of the solution work as follows:

1. AWS Amplify web application with static pages hosted on Amazon S3 and CloudFront serves content via HTTPS protocol.
2. Amazon Cognito provides user authentication.
3. After authorization, users can upload files to Amazon S3.

Prerequisites

• AWS account
• AWS Cloud9 IDE instance with 8 GiB memory, 2vCPUs, and 20 GB of Amazon EBS
• AWS Amplify CLI

Setting up the environment

We deploy this solution in the AWS Region us-east-1. If you want to run it in a different Region, make sure that the services Amazon Cognito and Amazon S3 are available. This step describes how to create a new AWS Cloud9 IDE and how to build the solution within it.

Note: AWS Cloud9 IDE instance is used only once to deploy the solution. It will turn itself off after 30 minutes idle.

To get started, you must first sign in to your AWS account. Now, to create the AWS Cloud9 IDE, follow these steps:

1. In the AWS Management Console, search for AWS Cloud9.
2. Select Create environment.

Figure 2: Creating an AWS Cloud9 environment

3. Enter the Environment name and description and click Next step button.
4. Configure the Environment as shown on the following screen:
   • Environment type: Create a new EC2 instance for environment (direct access)
   • Instance type: Other instance type (t2.large)
   • Platform: Amazon Linux 2 (recommended)
   • Cost-saving setting: After 30 minutes (default)
   • IAM role: AWSServiceRoleForAWSCloud9

Figure 3: Configuring AWS Cloud9 environment

5. Leave all the rest in default and select Next.
6. Review the summary and click Create the environment. It may take a couple of seconds until the environment is created.

After the environment is created, you will see an AWS Cloud9 welcome screen with a command line terminal.

Figure 4: AWS Cloud9 terminal

Increase AWS Cloud9 instance volume

A typical AWS Cloud9 instance comes with 10 GB of disk space where 8 GB is consumed by the OS installation running on it, this is not sufficient to download and build the application, you must expand the Amazon EBS volume to support the requirements to build the app. An EBS volume with additional 10 GB satisfies the requirement.

1. Copy and paste the following command in the AWS Cloud9 terminal:

curl https://raw.githubusercontent.com/aws-samples/s3uploader-ui/main/cloud9_resize.sh > cloud9\_resize.sh

Figure 5: Downloading script to extend the volume

2. Run the script by issuing the following command on the AWS Cloud9 terminal. This will expand the volume to 20 GB:

sh cloud9\_resize.sh 20

3. Reboot AWS Cloud9 instance to take volume increase into effect.

sudo reboot

4. Wait a few seconds, and the reboot will automatically reconnect you to Cloud9 instance. Once connected, check the free space using the following command:

df -h

Figure 6: Output of the command df -h

Note: Depending on the instance type, you may see a different disk, such as /dev/nvme0n1p1.

Building the web application

Complete the following steps to implement this solution:

1. Open AWS Cloud9 IDE and add a configuration file to specify the default AWS Region by copying the following command and pasting it into the AWS Cloud9 terminal:

cat <<END > ~/.aws/config
[default]
region=us-east-1
END

2. Create the AWS Amplify application by running the following command in the AWS Cloud9 terminal:

git clone https://github.com/aws-samples/s3uploader-ui.git

When you create the app, it creates a folder structure similar to the following image:

Figure 7: View of the AWS Cloud9 IDE file folder structure

3. Install and configure the AWS Amplify CLI.

Note: It is recommended to run this command from the root of your app directory. In this example, it is "s3-uploader-ui." For the next steps, if you receive a deprecation warning messages such as, 'npm WARN deprecated,' just ignore it. If you receive a message such as, "(node:19991) [DEP0128] DeprecationWarning: Invalid 'main'," just ignore and press enter to continue.

cd s3uploader-ui

npm install -g @aws-amplify/cli

4. Make sure that AWS Amplify CLI was installed by typing the following command:

which amplify
Output: ~/.nvm/versions/node/v16.13.1/bin/amplify

Note: Depending on the instance type, you may see a different version.

5. Initialize the project.
   • In the AWS Cloud9 terminal, inside the app directory, type the command:

amplify init

• • Select the following parameters:
    • Enter a name for the project: s3-uploader-ui (it can be any name; if you wish, you can leave defaults). Press enter.
    • Initialize the project with the above configuration: Yes. Press enter.
    • Select the authentication method you want to use: AWS profile. Press Enter.
    • Please choose the profile you want to use: default. Press Enter.

Figure 8: Amplify init output after answering the questions

Adding authentication to the web application

Add Amazon Cognito authentication to the AWS Amplify application with the amplify add auth command. 

1. In the AWS Cloud9 terminal, inside the application directory, type the command:

amplify add auth

2. Select the following parameters:
   • For Do you want to use the default authentication and security configuration?, select Default Configuration. Press enter.
   • For How do you want users to be able to sign in?, select Username. Press enter to confirm.
3. For Do you want to configure advanced settings? Select No, I am done.

Figure 9: View of the command amplify add auth after answering the questions

Adding storage resource to the web application

In this step, we create a bucket to allow authenticated users to upload files.

1. In the AWS Cloud9 terminal, inside the application directory, type the command: amplify add storage.
2. Select the following parameters:
   • For Select from one of the below mentioned services, select Content (Images, audio, video, etc.). Press enter to confirm.
3. Provide a friendly name for your resource that will be used to label this category in the project - for example: s35e505e53 (it can be any name; if you wish, accept the defaults). Press enter.
4. Provide bucket name. This is the bucket where users will upload files. For example: s3uploaderxxxxx. The name must be unique; otherwise, accept the defaults suggested and select enter to confirm. Make a note of this bucket; you use it later.

1. • Who should have access: Select Auth users only, use arrow key to move between the options and hit enter to select.
   • What kind of access do you want for Authenticated users? Use your arrow key to pick create/update and then hit the space bar to select it. Select enter to confirm.
   • Do you want to add Lambda Trigger for your S3 Bucket? Select No and press enter to confirm.

Figure 10: Screen of the command amplify add storage after answering the questions

Add the web hosting for the application

Add web hosting with the amplify hosting add command, it creates a bucket to store the static content of the application.

1. In the AWS Cloud9 terminal, inside the application directory, type the command: amplify hosting add.

2. Select the following parameters:
   • For Select the plugin module to execute, select Amazon CloudFront and S3. Press enter to select.
   • For Select the environment setup, select PROD then press enter to confirm.
   • For hosting bucket name, select s3uploaderui-2022-hostingbucket. This name should be unique as well; you can accept the suggested name. This is another bucket that will be created automatically to host the application's static webpages.

Note: Selecting PROD creates a CloudFront distribution providing secure access to the application via HTTPS.

Figure 11: Amplify hosting add after answering the questions

Install all modules and dependencies necessary to initialize your application. In the AWS Cloud9 Terminal, inside your application directory, type the command:

npm install

Figure 12: Example of output from the command npm install

In the previous screen, we just confirmed what we need for the application. To create the selected resources in AWS, use the command amplify push. In the AWS Cloud9 terminal, inside your application directory, type the command:

amplify push

For Are you sure you want to continue? Enter Yes and press enter to confirm.

Note: It may take few minutes to create the resources

Figure 13: Example of the output from the command: amplify push

It is now time to publish the application and make it publicly available. In this example, we are publishing the front-end component to Amazon S3. Enter the following command in the AWS Cloud9 terminal, inside your application directory:

amplify publish

Note: Make a note of the URL created in the end of this command. This will be used to access your application.

Figure 14: Screen of the command amplify publish output with the application URL

Testing the application

1. In a browser, navigate to the public URL using the output URL from the previous command.

Figure 15: Sign-in screen when you open the application URL in the browser

2. Sign up users.

When you navigate to the application URL, you'll only see a Sign in button to sign in to the application. For security reasons, we hide the Sign up button to prevent anyone from registering as a user. If you want everyone to be able to see the sign-up (Create Account) tab, like the following screenshot, proceed with the following steps. Otherwise, if you prefer leaving the sign-up tab hidden, jump to step 3 (creating authenticated users manually).

Figure 16: Sign In and Create Account tabs

2.1. In the AWS Cloud9 console, navigate to the folder path: s3-uploader-ui/src/. Open the file App.css.

• In the file, go to line 15 as shown in the following screen:

Figure 17: Exhibiting the sign up button in the Amazon Cognito authentication page

2.2. Remove the slashes at the beginning and at the end of the line. It will look like the following:

Line 15: 
*This CSS hide the signUp tab

• Save the file, and use the following commands to display the Create Account tab.

amplify push
amplify publish

Return to the application URL and check if you can see the Create Account tab.

Figure 18: Sign In and Create Account tabs

3. Manually create authenticated users.

If you want to keep the application with only the sign-in interface, you can add users using the AWS Management Console. To do this, follow these steps:

• Go to the Amazon Cognito console and select Manage User Pools. Select the user pool created by AWS Amplify; it may start with S3uploadxxxxxxx_userpool_xxx-dev.
• Under General settings, select Users and Groups.
• Under the User tab, select Create user.
• Fill in the Username, Temporary password, and Email. Uncheck the box next to Mark phone number as verified?

Figure 19: Create user from the Amazon Cognito console

4. Sign in to the web application.

Now that you have signed up a new user in your application, sign in with the credentials created earlier.

After signing in to the application, you will see the user interface with a button Choose file(s). Select this button and select one or more files to upload. After you have selected the files, select Upload to send those files to S3.

Figure 20: Web application with multiple files selected

Note: The front end does not perform any file validation in terms of size or type, but it is possible to customize the solution to include extra validation of the data before it is uploaded to Amazon S3.

To validate if the files uploaded were stored in the S3 bucket, first go to the AWS Management Console and select the Amazon S3 service. Select the bucket created by the AWS Amplify application to host your files. This is the bucket created in step 4 of this blog ("Adding storage resource to the web application").

Figure 21: AWS Management Console showing the S3 objects

Note: If you forgot to note the bucket name, you can find the name in the CloudFormation output that AWS Amplify generated.

• Go to the AWS CloudFormation console. In the left corner, select Stacks.
• Enable View nested as in the following screen:

Figure 22: Enabling view stack

• There will be four stacks created for the project. All of them have the same initials. Select the ones that have “storages” in the name. For example, (amplify-s3uploaderui-dev-151739-storages).
• Navigate to the Outputs tab in the stack selected. The BucketName value in the stack is where the files are uploaded.
• Now that you know the bucket name, access the S3 bucket in the AWS Management Console to check the files that have been uploaded.

Figure 23: Bucket where the uploads are saved

Cleaning up

Now that you have finished this walkthrough solution, you can delete your AWS Amplify application if you aren’t going to use it anymore.

Note: This action cannot be undone. Once the project is deleted, you can’t recover it. If you need it again, you have to re-deploy it.

1. To delete the application from AWS, run the following command from the terminal:

amplify delete

Figure 24: The screen above displays the results of the amplify delete command

2. Delete your AWS Cloud9 instance.

Conclusion

In this post, we demonstrated how to build a serverless application to support file uploading to Amazon S3 by external users. Using just a few AWS services, such as AWS Amplify, Amazon S3, Amazon CloudFront, and Amazon Cognito, you can easily build a web application to store files securely to Amazon S3. The solution provides authenticated access using a hosted UI to sign in and sign up users, built using an Amazon Cognito-hosted UI.

With this solution, maintenance and scalability issues can be replaced with managed services in a quick and secure way. You can provide external users an easy and secure way to get their files into your S3 buckets, all without providing them access to your buckets. This can help you lower the threat of any security compromises, while still enabling you to use external data to help further your business goals or meet business demands.

Thanks for reading this blog post, if you have any comments or questions, feel free to leave them in the comments section.