DEV Community

Cover image for Run Batch Jobs with the Code Engine CLI
IBM Developer profile image Jenna Ritten
Jenna Ritten for IBM Developer

Posted on

Run Batch Jobs with the Code Engine CLI

What is a batch job?

A batch job is a scheduled program that is assigned to run on a computer without further user interaction. Batch jobs are often queued up during working hours, then executed during the evening or weekend when the computer is idle.

Once the batch job is submitted, the job enters into a queue where it waits until the system is ready to process the job. If the job queue contains many jobs waiting to be processed, the system processes the jobs either in chronological order or by priority.

Batch jobs are frequently used to automate tasks that need to be performed on a regular basis, like payroll, but don’t necessarily need to occur during the day or have an employee interacted with the system. Jobs that happen on a regular basis are incorporated into batch schedules.

A job runs one or more instances of your executable code. Unlike applications, which handle HTTP requests, jobs are designed to run one time and exit. When you create a job, you can specify workload configuration information that is used each time that the job is run.

The most valuable benefits of batch jobs include:

  • Large programs can utilize more dedicated servers when the work processes are in night mode.
  • With fewer users or employees online during off hours, the performance will be faster and more efficient at night. During the day it could be restricted to fewer servers and numbers needed.
  • Employees are freed up to focus on less repeatable, more creative tasks.

The following guide will walk you through creating a Code Engine project and application using sample code, and then you will create batch jobs and run them. The following steps can also be followed to deploy an application to Code Engine from source code after following the setup instructions in Next Steps.

Happy Hacking!

Let's Build Something

Steps

  1. Sign Up for a Free IBM Cloud Account
  2. Install the IBM Cloud Developer Tools CLI (command line interface, the commands you type into your Terminal)
  3. Install the IBM Cloud Code Engine Plugin
  4. Login to Your IBM Cloud Account via the CLI
  5. Create a Code Engine Application
  6. Create a Job from a Public Repo
  7. Run a Batch Job
  8. NEXT STEPS! Build a Container Image from Source Code to Deploy to Code Engine and Run Batch Jobs

Setup & Installation

2. Install the IBM Cloud Developer Tools CLI

  • For Mac and Linux, run the following command in Terminal: 
curl -sL https://raw.githubusercontent.com/IBM-Cloud/ibm-cloud-developer-tools/master/linux-installer/idt-installer | bash
Enter fullscreen mode Exit fullscreen mode
  • Verify the IBM Cloud Developer Tools CLI is installed 
ibmcloud dev help
Enter fullscreen mode Exit fullscreen mode
  • For Windows 10 Pro, run the following command as an administrator in Powershell: 
[Net.ServicePointManager]::SecurityProtocol = "Tls12, Tls11, Tls, Ssl3"; iex(New-Object Net.WebClient).DownloadString('https://raw.githubusercontent.com/IBM-Cloud/ibm-cloud-developer-tools/master/windows-installer/idt-win-installer.ps1')
Enter fullscreen mode Exit fullscreen mode

Note For Windows Users: If you encounter a Git error similar to the one below, you will need to install Git in the correct path.

bash: git: command not found
Enter fullscreen mode Exit fullscreen mode

Follow the Windows Guide HERE

3. Install the IBM Cloud Code Engine Plugin

  • For Mac, Linux, and Windows 10 Pro, run the following command: 
ibmcloud plugin install code-engine
Enter fullscreen mode Exit fullscreen mode
  • Verify the IBM Cloud Code Engine Plugin is installed 
ibmcloud ce help
Enter fullscreen mode Exit fullscreen mode

4. Login to Your IBM Cloud Account via the CLI

  • For Mac, Linux, and Windows 10 Pro, run the following command: 
ibmcloud login
Enter fullscreen mode Exit fullscreen mode

  • Enter email and password

  • View available resource groups

ibmcloud resource groups
Enter fullscreen mode Exit fullscreen mode
  • Assign a target resource group (default to your "Default") 
ibmcloud target -g Default
Enter fullscreen mode Exit fullscreen mode

Deploy to Code Engine & Run Batch Jobs

5. Create a Code Engine Application

  • Create a new Code Engine project and give it a name 
ibmcloud ce project create --name PROJECT_NAME
Enter fullscreen mode Exit fullscreen mode 
ibmcloud ce project create --name sandbox
Enter fullscreen mode Exit fullscreen mode
  • Create a new app from a sample Container Image 
ibmcloud ce application create --name APP_NAME --image IMAGE
Enter fullscreen mode Exit fullscreen mode 
ibmcloud ce application create --name myapp --image docker.io/ibmcom/hello
Enter fullscreen mode Exit fullscreen mode
  • Check the application status 
ibmcloud ce application get -n APP_NAME
Enter fullscreen mode Exit fullscreen mode 
ibmcloud ce application get -n myapp
Enter fullscreen mode Exit fullscreen mode
  • Get the live application URL 
ibmcloud ce application get -n APP_NAME -output url
Enter fullscreen mode Exit fullscreen mode 
ibmcloud ce application get -n myapp -output url
Enter fullscreen mode Exit fullscreen mode
  • View the live application at the URL in your browser

6. Create a Job from a Public Repo

  • Create a job configuration that is named myjob and uses the container image docker.io/ibmcom/firstjob 
ibmcloud ce job create --name JOB_NAME  --image IMAGE
Enter fullscreen mode Exit fullscreen mode 
ibmcloud ce job create --name myjob  --image ibmcom/firstjob
Enter fullscreen mode Exit fullscreen mode

Note: the format for creating a new job from an image is ibmcom/firstjob from the image docker.io/ibmcom/firstjob.

7. Run a Batch Job

  • Run a job 
ibmcloud ce jobrun submit --name testjobrun --job myjob --array-indices "1 - 5"
Enter fullscreen mode Exit fullscreen mode

Note: the following jobrun submit command creates five new instances to run the container image that is specified in the myjob job. the resource limits and requests are applied per instance, so each instance gets 4 G memory and 1 vCPU. This job allocates 5 * 4 G = 20 G memory and 5 * 1 vCPU = 5 vCPUs.

  • Resubmit a job run based the configuration of a previous job run, use the jobrun resubmit command 
ibmcloud ce jobrun resubmit --jobrun testjobrun
Enter fullscreen mode Exit fullscreen mode
  • Access the Job Details, including the status of your instances, configuration details, and environment variables of your job 
ibmcloud ce job get --name myjob
Enter fullscreen mode Exit fullscreen mode

Options for Creating and Running Jobs

CONGRATULATIONS!

If you've made it this far, then Kudos to you! The example commands given can be followed to create an application from a container image and deploy it to Code Engine with your own images. You can also follow the example commands given to create and run batch jobs with your own public images and repositories. For more information to get started with your own container images and/or source code, check out the Tutorial Tuesday Code Engine Guide along with the Code Engine Documentation.

8. Next Steps

1. Build a Container Image from Source Code

2. Plan a Container Image for Code Engine Jobs

3. Create a Job from a Public Repository

4. Run a Batch Job!

Top comments (0)

Read next

getambassador2024 profile image

From Basics to Real-Time Performance: a Complete Guide to API Requests

Ambassador -

neehar_priydarshi_4a16d92 profile image

Implementing Retrieval-Augmented Generation with LangChain, Pgvector and OpenAI

neehar priydarshi -

abhinowww profile image

Mistral vs GPT: A Comprehensive Comparison of Leading AI Models

Abhinav Anand -

_e7530225be583f1 profile image

"IT" - nima degani?

Абдумомин -