How to Run a Python Script in the Cloud

Eddie Svirsky — Mon, 15 Jul 2024 23:17:30 +0000

Python is fantastic for quick and easy development, with plenty of tutorials available to get you started on your computer. However, there are times when you need to run your script in the cloud—maybe you're away from your PC, need to share the script with someone else, or want the script to reside on a server so it can be accessed by other users or services at any time.

Let’s explore some simple ways to get your Python script up and running in the cloud.

Google App Engine

One option you have is to use a Platform-as-a-Service (PaaS) service. A PaaS is a cloud computing model that provides a platform allowing customers to develop, run, and manage applications without the complexity of building and maintaining the underlying infrastructure. There are several PaaS options available, but let's take a look at Google App Engine, which is a fully managed serverless platform that allows you to build and deploy applications at scale.

Steps to Deploy a Python Script on Google App Engine

First, you need to create a project in Google Cloud Platform (GCP) and set up the Google Cloud SDK.

Create a Project:
- Go to the GCP Console.
- Click on the project drop-down and select "New Project."
- Enter a name for your project and click "Create."
Set Up gcloud:
- Download and install the Google Cloud SDK
- Initialize the SDK by running the following command in your terminal:
```
gcloud init
```
- Follow the prompts to authenticate and set your project.

Create a new directory on your local machine for your project files.

mkdir python-script
cd python-script

Create a Python script (main.py):

Flask is a lightweight web framework for Python. Create a simple Flask app that we can deploy.

from flask import Flask
import requests
from bs4 import BeautifulSoup

app = Flask(__name__)

@app.route('/')
def get_google_title():
    response = requests.get('https://www.google.com')
    soup = BeautifulSoup(response.text, 'html.parser')
    title = soup.title.string
    return f'Title of Google page: {title}'

if __name__ == '__main__':
    app.run(host='0.0.0.0', port=8080)

Create app.yaml:

The app.yaml file is used to configure the App Engine deployment settings.

runtime: python312

handlers:
- url: /.*
  script: auto

Create requirements.txt:

List the dependencies for your project in requirements.txt.

Flask==3.0.3
requests==2.32.3
beautifulsoup4==4.12.3

Deploy the application:

Deploy your application using the gcloud command.

gcloud app deploy

After deployment, you'll get a URL where your application is hosted. Visit this URL to see your running application.

gcloud app browse

This command will open your default browser and navigate to your app's URL.

Billing and Costs

A PaaS is billed based on the resources your application consumes. With Google App Engine, you are charged for the instance hours, storage, and bandwidth your application uses. By default, it sets you up with an F1 instance in the Standard Environment, and you are charged based on your usage. It can get pretty complicated, but for full pricing details, you can visit the Google App Engine pricing page.

AWS Lambda

Another option is to use serverless computing, which allows you to run code without provisioning or managing servers. Unlike PaaS, with serverless all infrastructure management is abstracted away, you don’t have to deal with instances at all.

Steps to Deploy a Python Script on AWS Lambda

First, you'll need to create an AWS account and set up the AWS Command Line Interface (CLI).

Create an AWS Account:
- Go to the AWS website and create a new account.
Access AWS Lambda:
- Sign in to the AWS Management Console.
- From the AWS Management Console, navigate to the AWS Lambda service by typing "Lambda" in the search bar and selecting it.
Create Function:

- Click the "Create function" button.
- Select "Author from scratch."
- Provide a function name (e.g., python-script).
- Choose "Python 3.12" as the runtime.

Click "Create function":
- This will create a new Lambda function with a basic execution role.

To use external libraries such as requests and beautifulsoup4, you need to package your dependencies.

Create a Deployment Package with Dependencies:

On your local machine, create a directory for your Lambda function and navigate to it:
```
mkdir python-script
cd python-script
```

Set up a virtual environment and install the required libraries:

python3 -m venv venv
source venv/bin/activate
pip install requests beautifulsoup4

Create a lambda_function.py file with the following content:

import requests
from bs4 import BeautifulSoup

def lambda_handler(event, context):
    response = requests.get('https://www.google.com')
    soup = BeautifulSoup(response.text, 'html.parser')
    title = soup.title.string
    return {
        'statusCode': 200,
        'body': f'Title of Google page: {title}'
    }

Package your function and its dependencies:

deactivate
cd venv/lib/python3.8/site-packages
zip -r9 ../../../../function.zip .
cd ../../../../
zip -g function.zip lambda_function.py

Upload the Deployment Package:
- In the AWS Lambda console, go to your function.
- Under "Function code", select "Upload a .zip file" and upload function.zip.
Click "Deploy":
- After editing the code, click the "Deploy" button to save and deploy your changes.

If you want to expose your Lambda function via HTTP, you can set up an API Gateway.

Create an API Gateway:

- Go to the API Gateway console.
- Click "Create API" and select "HTTP API."
- Follow the prompts to set up a new API and create a route that triggers your Lambda function.

Integrate Lambda Function:
- Choose your Lambda function as the integration target.
- Deploy the API to get a public endpoint URL.

Once your Lambda function is set up, you can invoke it using the provided URL from API Gateway or directly from the AWS Management Console.

Billing and Costs

AWS Lambda is billed based on the number of requests and the duration of your code execution, measured in milliseconds. You get a generous free tier, which includes 1 million free requests and 400,000 GB-seconds of compute time per month. For detailed pricing information, visit the AWS Lambda pricing page.

Codeupify

Another serverless option is Codeupify, a service designed to make it very easy to deploy functions into the cloud. Using Codeupify you can quickly set up, deploy, and manage your scripts through a user-friendly interface.

Steps to Deploy a Python Script on Codeupify

Sign Up:
- Go to codeupify.com and sign up for a new account.
Create a Function:
- Name: python-script
- Description: Our cloud python script
- Language: Python
- Image: python:3.12.1-slim
- Concurrency: sync
- Packages:
  - Select requests to enable API queries from our code
  - Select beautifulsoup4 to be able to parse HTML
- Create Function: Click on the "Create" button
Add your code
- In the function editor, you can write or paste your Python code.

import requests
from bs4 import BeautifulSoup

def handler(request):
    response = requests.get('https://www.google.com')
    soup = BeautifulSoup(response.text, 'html.parser')
    title = soup.title.string
    return title

Save

You should see a URL appear in the top right corner of the screen. This is the endpoint where your function is deployed and can be accessed.

Conclusion

Each solution has its strengths and weaknesses. Google App Engine offers more control and customization of your environment, which is great for applications that need specific configurations and integrations but may require more management effort. With AWS Lambda, you don’t have to worry about managing servers at all, as it automatically scales with demand. However, you might encounter cold starts and need to package dependencies carefully. Codeupify provides an exceptionally user-friendly interface, making it the easiest to deploy your Python scripts quickly, but it might not offer the same level of customization as Google App Engine.

Regardless of which solution you choose, you can get your Python script running in the cloud pretty quickly.

DEV Community: Eddie Svirsky

How to Run a Python Script in the Cloud

Google App Engine

Steps to Deploy a Python Script on Google App Engine

Billing and Costs

AWS Lambda

Steps to Deploy a Python Script on AWS Lambda

Billing and Costs

Codeupify

Steps to Deploy a Python Script on Codeupify

Conclusion