DEV Community: Eswari Jayakumar

Access and Upload S3 Private Bucket Files Without Credentials

Eswari Jayakumar — Sun, 30 Jul 2023 16:04:13 +0000

Normally, S3 buckets are set to private, meaning you need to grant access to individual users in order for them to access their files. However, there may be situations where certain files in the S3 bucket need to be accessed by users without requiring any credentials or permissions.

Looking for a solution? AWS S3 offers pre-signed URLs, a highly beneficial feature to address your needs.

Here are some scenarios that illustrate the importance of using presigned URLs:

If you're creating a web conferencing software and saving video recordings in a private S3 bucket, you may need to grant access to a specific recording to an individual user. In that case, you can create a presigned URL for that single file and share the URL with the user.
In your product, users upload profile pictures on their profile page. To upload these pictures directly to the S3 bucket from the front end, you'll need to use presigned URLs.

Benefits of Presigned URLs:
Limited Time Access: You can configure the URLs' expiration time when creating them. So, it can be useful only for a limited period of time.

Safe Access of Buckets: As it is possible to expose only relevant files to the public, there is no need for you to change the entire bucket to be publicly accessible. Your S3 bucket will stay private always.

Secured URLs: Nobody can alter the URL parameters when uploading the files to the S3 bucket. This is because the URLs are created using cryptographic signatures and the verification will fail when someone tries to change it.

Now, Let's learn about programmatically creating pre-signed URLs using the AWS Boto3 SDK.

In your project folder, Install the Boto3 SDK by running the command.

pip install boto3

Accessing the Private Objects in the S3 bucket
Then you can create a Python file inside your folder, copy and execute the below code.



import boto3

session = boto3.Session(
    aws_access_key_id="<Provide your AWS access Key ID>",
    aws_secret_access_key="<Provide your AWS Secret Key>",
    region_name="us-east-1"
)

s3_client = session.client('s3')

url = s3_client.generate_presigned_url(
    ClientMethod="get_object",
    Params={
        'Bucket': 'demobucket',
        'Key': 'test-cover.png'
    },
    ExpiresIn=1800
    )

print(url)

This is what happens:

Create a Boto3 Client for s3 using your AWS Credentials.
Using the S3 Client, Generate a presigned URL for the test-cover.png object in the bucket named "demobucket".
Here the expiry is set as 1800 seconds. But you can alter it as you want.

On executing the code, you will get an URL like below:

https://demobucket.s3.amazonaws.com/test-cover.png?AWSAccessKeyId=AKIAY********&Signature=%2FfE94is0ONgAVW4LgafRLWxgvLg%3D&Expires=1690732722
You can share this URL with any of your users or web applications. With this URL, users can easily view the objects without needing any extra credentials or access.

Uploading an Object to a Private S3 bucket
In order to upload files from your application's frontend to private S3 buckets, you will need to generate a presigned post URL.




response  = s3_client.generate_presigned_post(
    Bucket='demobucket',
    Key='testimage.png',
    ExpiresIn=1800
)

print(response)

Use the above code to generate a post URL for uploading an object by providing the desired Bucket Name, Object Name and Expiration time.

If everything works well, you will get the JSON response as follows.




{
   "url":"https://demobucket.s3.amazonaws.com/",
   "fields":{
      "key":"testimage.png",
      "AWSAccessKeyId":"AKIAY*******",
      "policy":"eyJleHBpcmF0aW9uIjogIjIwMjMtMDctMzBUMTY6MDU6MThaIiwgImNvbmRpdGlvbnMiOiBbeyJidWNrZXQiOiAiZXN3YXJpYnVja2V0In0sIHsia2V5IjogInRlc3RpbWFnZS5wbmcifV19",
      "signature":"qbpmLnc4pBaVFSRlXzcsrQmYXRw="
   }
}

The response will contain the fields such as AWS Access Key ID, Policy and Signature.

Here I have tried to upload a file using the pre-signed URL from Postman. You can even use it directly in your app UI where you have the file upload functionality.

To properly send your request to the URL, ensure that you set the fields and files as form data in the body of the request.

This will upload the file to the S3 bucket.

Conclusion

In this blog post, you explored how to interact with Amazon S3 using Boto3, the AWS SDK for Python.

We started by generating a pre-signed URL for an object in a private S3 bucket, allowing users or web applications to easily view the object without requiring any extra credentials or access.

Next, we delved into the process of uploading an object to a private S3 bucket from an application's frontend.

Happy Coding !

Automate Ubuntu User creation with SSH Key access using a shell script

Eswari Jayakumar — Sun, 02 Jul 2023 17:46:34 +0000

This blog will teach you about the shell script to automate new user creation in Ubuntu with administrative privileges. Also, you will learn about how to set up SSH access with an authorized key for the same user.

1. Generate an SSH key in the local machine
Before executing the script in the target machine, Generate an SSH key in the local machine from where you are planning the access the target machine.

In your local machine, Open Terminal and execute the below command



ssh-keygen -t rsa

Now, Execute the command and generate public and private keys. As per the above screenshot, the public key gets saved in the file testuserrsa.pub. Copy the contents of the file and provide it in the below script for the variable PUBLIC_KEY

2. Execute the script in the target machine
Below is the script which is used to automate Ubuntu user creation.

Let us see a detailed line-by-line explanation of the commands.



sudo adduser --disabled-password --gecos "" testuser

This command will create a new user named “testuser” without setting any password ( — disabled-password). It does not ask for any additional information(--gecos “”) and creates a user with default settings.



sudo usermod -aG sudo testuser

This command simply adds the user to the “sudo” group which grants admin privileges to the user. So, the user can run commands with root privileges using the “sudo” command.



sudo mkdir -p /home/testuser/.ssh && 
sudo touch /home/testuser/.ssh/authorized_keys

By this command, a new directory named .ssh is created inside the home directory of the user. i,e. /home/testuser. Then, a new file named “authorized_keys” is created within the .ssh directory.



sudo chmod 700 /home/testuser/.ssh && 
sudo chmod 600 /home/testuser/.ssh/authorized_keys

With this command, you are setting appropriate permissions for the .ssh directory and authorized_keys file. Using chmod 700 command, the owner will have all the permissions over the directory. Using chmod 600 command, the Owner will have read and write permissions for the authorized_keys file, no access for the group or everyone else.



sudo chown -R testuser /home/testuser/.ssh

This command changes the ownership of the .ssh folder to the user “testuser”.



sudo sh -c "echo $public_key > /home/testuser/.ssh/authorized_keys"

With the help of this command, the public SSH key is written into the authorized_keys file in the .ssh folder of testuser’s home directory.



sudo sh -c "echo 'testuser ALL=(ALL) NOPASSWD:ALL' > /etc/sudoers.d/testuser-user"

On executing this command, this creates a new file “testuser-user”inside /etc/sudoers.d directory and makes the entry “testuser ALL=(ALL) NOPASSWD:ALL” inside that file. In this way, the testuser will be granted permission to execute any command with sudo privilege without requiring any password.

Download the above script file and execute it either manually or using any automation tools, the user will be created without any external user prompts.

In this way, you can automate Ubuntu user creation with admin privileges and SSH key access.

Thanks for reading !

If you like this article, do follow me and connect with me in LinkedIn to learn about more interesting topics.

How to send email with attachments in Node.js using Nodemailer

Eswari Jayakumar — Tue, 29 Sep 2020 03:31:45 +0000

Hello everyone !

In this tutorial, We are going to learn about how to send emails in Node.js using the Nodemailer module just simply with gmail.

Prerequisites:

A Gmail account
Basic knowledge of JavaScript and NPM(Node package Manager)

Install Nodemailer and Import it in source code:

To start with, create a working directory. Open command prompt and navigate to the directory and run the command npm init -y. A package.json file is generated inside the folder.

Then Run the npm install nodemailer -s command to install the Nodemailer package.

Nodemailer is a module in Node.js to send emails easily.

After installing the Nodemailer, create a file send-mail.js inside the same working directory. Import the module inside the file.

const nodemailer = require('nodemailer');

Configure Gmail account

Create a Nodemailer transporter object by providing the details of the email account.

let transporter = nodemailer.createTransport({
    service: 'gmail',
    auth:{
        user: 'xyz@gmail.com',
        pass: 'xxxxxxx'
    }
});

Set the service as gmail and provide your email address and password in 'auth object'.

Gmail provides a concept of Less secure apps using which we can use plain password to send emails. Turn on this setting in the link https://www.google.com/settings/security/lesssecureapps.
Instead of directly providing the password, we can also use OAuth2 by adding Oauth token details in the transporter object.

Set the Email contents

Next create a mailContent object with necessary details for sending an email such as from address, to address, subject, mail body content and attachments.

let mailContent={
    from: 'Sender Name <xyz@gmail.com>',
    to: 'Receiver Name <receivername@gmail.com>',
    subject: 'First Node.js email',
    text: 'Hi,This is a test mail sent using Nodemailer',
    html: '<h1>You can send html formatted content using Nodemailer with attachments</h1>',
    attachments: [
        {
            filename: 'image1.jpg',
            path: __dirname + '/image1.jpg'
        }
    ]
};

The parameters provided inside the mailContent object are:

from - Provide the sender name and email address. This should be same as the user email configured in the transporter object above.
to - Provide the recipient name and email address
subject - Provide the email subject
text - Configure the plain text email content
html - If we want to send a proper html formatted mail, we can provide the html contents in this parameter.
attachments - In this parameter, We can include a list of attachments which needs to be sent along with the mail.

Send Emails:

As we have completely configured the mail details, now we can send the mail using the sendMail method in the transporter object.

transporter.sendMail(mailContent, function(error, data){
    if(err){
        console.log('Unable to send mail');
    }else{
        console.log('Email send successfully');
    }
});

In the above code snippet, I have provided the mailContent paramter as input for the sendMail method.

We are done with the necessary coding part. Navigate to Command prompt and execute this file by running the command node send-mail.js.

Woohoo ! Mail is delivered in recipient's inbox.

Happy Coding !!!