DEV Community: Codeitout

Spring Batch Part 2 for Beginners- The Domain Language of Spring Batch

Codeitout — Sat, 12 Nov 2022 09:20:38 +0000

Let's start with this diagram which highlights the key concepts which make up the domain language of spring batch.
Let's start understanding each of these concepts in detail.

Job

It is an entity which encapsulates entire batch process. It is wired together with the help of some configuration file (XML/Java based) which is called as job configuration.
Job is on top of the hierarchy.It is a container for Step instances.

This diagram will be explained in subsequent steps .
Job contains multiple steps in it and orders up the steps in the order of their execution. It applies the configuration globally on all steps like the restorability

The Job Configuration contains

Name of Job
Defination and ordering of step instances.
restartability of Job

Here if we see "footballJob" is the Job name , playerLoad(), gameLoad(), playerSummarization() are all steps .
.start() tells the job to start the playerLoad() step and once its completed .next() tells the Job to run the gameLoad() step and once it's completed playerSummariztion() step will be executed.

Job Instance

It refers to the concept of logical Job run.Let's understand in simple words.
Suppose you have a EOD job to fill the data from 1 table to other table [based on today's date]. There is only one EOD Job. But each day you have to run the Job with it's own parameters. In the case of this job, there is one logical Job Instance per day. There is Jan 1 run,Jan 2 run and so on. If the Jan 1 run fails, we trigger it again- Its still the Jan 1 run.
Here we saw that a Job instance can have multiple executions.[Job executions]. But only a single Job instance wrt the Job with it's set of parameters can be run at a given time. Starting a new Job instance means start from the beginning, whereas using an existing instance means starting from where we left off.

Job Parameters
How do we distinguish one Job instance from the other?
It holds a set of parameters which is used to start a Job. Something similar to passing arguments to a function.In the earlier case where we had a Job instance for Jan 1 and one more for Jan 2, It is a single job but it is differentiated based on the Job parameters.
Jan 1 would have started based the date parameter as 1 Jan and Jan 2 run would have the date parameter as 2 Jan.

Job instance = Job + Identifying parameters

Job Execution
It refers to the concept of single attempt to run a job.Job instance can be executed multiple times. This is called as execution. The execution may result in failure or completion but the Job instance corresponding to an execution will be considered as complete only when that execution is successful. Wrt to the EOD job, it might be possible that the 1st execution of Jan 1 Job instance failed. If it was run again with same job parameters , a new Job execution is created.
There are few Job execution properties like status, startTime, endTime, exitStatus, executionContext etc. These properties are persisted and can be use to check the status of the Execution.

Just assume that the developer took that entire day to figure out the issue. The next window for the execution of Job opened up again at 9 pm and this time she has run the Job instance of 1 Jan and it's successful at 9:30pm. And since it is the second day we must run the Job for Jan 2 as well. So we started the Job instance at 9:31 pm and it ends at 10:30pm
There is no need to start the job instance one after the other if they don't access the same data otherwise they might end up in locking the db
Since 2 job instances are run, we should have one more entry in the INSTANCE table and 2 extra entries in Job execution table.

Step

A step is a domain object which encapsulates an independent, sequential phase of batch job. Each job may have multiple steps to facilitate the batch processing. It is totally dependent on the developer on how complex each step might be.
It can be as simple as fetching from db or complex as performing heavy operations on db using business logic, etc. Similar to Job, each Step has an StepExecution.

Step Execution
It represents single attempt to execute a step. A new Step Execution is created each time a step is run. If the step fails to execute because of the failing of step prior to it, no execution is persisted for it. Each step execution contains an ExecutionContext which contains data needed by the developer be persisted across the batch runs.
For eg state info needed to restart.
Some Step Execution properties are status, startTime, endTime, exitStatus, executionContext etc.

Execution Context
It represents a collection of key value pairs that are persisted and controlled by the framework in order to allow developers a place to store persistent data that is scoped to a step execution obj or a job execution object.
Eg: executionContext.putLong(getKey(LINES_READ_COUNT), reader.getPosition());

Let's suppose you are reading lines from a file. Here 'LINES_READ_COUNT' is the key and the value will be set by the developer through the code. This key value pair is stored in the metadata table which can be easily accessed accross the different steps of a Job.
In this case the step failed after processing 40322 lines allowing the step to start again from the same line where it left off.
Exactly 1 execution Context must be there per step execution. There should be at least 1 execution context per job execution per step execution.

Here, it checks if executionContext has LINES_READ_COUNT. If it's present then lineCount fetches it and stores the value corresponding to the Lines_Read_Count. It then starts reading data from the next lines in the file. Thus execution context saved the time + extra processing here as it has stored the data of already read lines.

Job Repository

It provides CRUD operations for Job launcher,Job and Step implementations.When a job is first launched, Job execution is obtained from the repository. And during the process of job execution and step execution, other data is also persisted in it which can be used by the developer. All the metaData tables are part of this repository. You can also change the default names of these tables.
@EnableBatchProcessing gives a JobRepository automatically configured.

Job Launcher

It is a simple interface for launching a Job with set of Job parameters.

public interface JobLauncher {
 public JobExecution run(Job job, JobParameters jobParameters)
 throws JobExecutionAlreadyRunningException, JobRestartException,
 JobInstanceAlreadyCompleteException, JobParametersInvalidException;
}

Item Reader

It is abstract class which represents retrieval of input for a step.When item reader has exhausted the retrieval of data, it returns null.

Item Writer

It is an abstract class representing output of a step, one batch or chunk at a time. It has no knowledge of pervious or next input which it takes. It knows only the item which was passed in the current invocation.

ItemProcessor

It is an abstraction which represents the business processing of data. It acts as a place of transformation of input data received from item reader to output of item writer.
If the data processed is not valid, it returns null which means the item writer won't include it while producing the output.

If you prefer watching video , you can follow this link🦋.

So now you know what to do. If you found this useful, you know what to do now. Hit that clap button and follow me to get more articles and tutorials on your feed.❤❤

The beginner guide to Spring Batch Processing- Part 1

Codeitout — Fri, 21 Oct 2022 19:41:11 +0000

Spring Batch is a Lightweight, compressive batch framework which is used to perform ETL tasks on bulk of data easily.

E- Extract
T-Transform
L-Load

It’s not a scheduler, but it is intended to work with scheduler. It can be used to load data from file to database , or read data from database, process it to give a different output and store them in a separate database. Basically it’s for large datasets or high volume of data.

Let’s dive into spring batch architecture.It mainly has 3 components.

Application — It contains batch jobs, code written by developers, APIS, etc.

Batch Core — It contains the core classes needed to launch and control the batch job i.e JobLauncher, Job, Step, JobExecution etc.

Infrastructure — It contains readers and writers, services which will be used by developers to code the batch job i.e ItemReader, ItemWriter, ItemProcessor etc.

Batch Processing Strategies

1.Normal processing in batch window
It is for simple batch processors. When application is updating the data, no other batch jobs or online users should be able to access it. This is the assumption. Single commit can be done at the end of the batch job.

2. Concurrent batch/Online Processing

This is used in cases where both the batch job and online users can use the data simultaneously. But the online user should not lock the data. Physical locking can be minimised by optimistic Locking and pessimistic locking approach. Commit is done at the end of few transactions each. Thus at a time only a small partition of data might not be available for other processes.

Optimistic locking

The usecase is at a given point the probability that both app and online users are updating the data is very less. Thus timestamp column is added into the table. When starting the batch job, it is noted down and while updating the data after processing it ,it checks the same timestamp in the where clause. It updates the data only if both the timestamp match. If the timestamp doesn’t match, it won’t update the data.

eg: UPDATE TABLE T1 SET C1 = ABC WHERE TIMESTAMP =T1

where T1 is timestamp which was read initially before starting the processing.

_Pessimistic locking
_
The usecase for it is when it’s highly likely that both app and online users are updating data at the same time. Physical/ logical locks need to be retrieved in these cases. Separate lock column can be maintained where when one user access the data, the lock is set true for the record and if some other user tries to access the same data at same time, he/she won’t be able to access it because of the lock.

3. Parallel Processing
It refers to multiple batch jobs running in parallel. Thus less time consumed. It can’t share the resources between each other[ 2 jobs running in parallel]. Data partitioning can be an option here. We can use multiple threads to process in parallel.We can also use control table which contains details of all shared resource and mentions if it is being used by some other application. Thus the same resource won’t be processed multiple times.

Cons — Control table can itself become a critical resource

4.Partitioning
It allows multiple versions of a batch application to run concurrently. Reduces elapsed time taken by long batch jobs. Database data/input data will be split, which allows the application to work on each data set independently.

Partioning != partioning the db [not necessarily]
Application should be flexible enough to decide the number of partitions. Automatic and user controlled config should can also be considered. It depends on the application developer writing the code.

Here in this diagram you can see that the input in split into multiple parts and is given to the controller. Thus these can run in parallel and the results are then merged to get the final output.

These are few partitioning approaches as well.

Fixed and even break up of record set where the input data will be split evenly thus each job batch will get same number of records.
Break up by a key column where the data will be split based on a column be it a date range or some id.
Break up by Views refers to breaking the record set into views.
Extract table to a flat file refers to extracting table records and exporting to a flat file based on size of file.

If you prefer watching video , you can follow this link🦋.

So now you know what to do. If you found this useful, you know what to do now. Hit that ❤️ button and follow me to get more articles and tutorials on your feed.❤❤

Why did you start coding? What were the changes that you saw in your life because of coding?

Codeitout — Wed, 24 Mar 2021 16:05:16 +0000

I have been asked this question by many people. So finally decided to write a post about it :p

When I was in first year of my engineering, I used to see lot of boys who used to code up cool stuffs. Some people even had their own startups. Well, at first didn't used to pay attention to what they did, but as you know you can't resist good things for a longer time.

I started learning C, C++ after that because I came from a non-CS background before engineering and I didn't even know the basics. After 2nd year, I did internship in image processing using python and OpenCV. That's when I really entered the tech world because my internship made me keen to explore various python libraries . Just imagine creating 100s of filters for a pic, extracting page numbers from a pic of 2 pages, identifying objects, tracking particular objects in video, making small games, creating your own mini paint application, writing scripts to automate the process of downloading youtube videos at night when u no longer use the mobile data, to automate the process of changing wallpapers daily on your desktop, playing with the text to speech conversion scripts ,so on… The list is never ending.
And guess who did all that ?

If you are guessing it's me, then you are absolutely right :D . For many developers out there,it might be nothing but for a girl who didn't even know the basics of coding,it's a pretty big deal😎.Then I started solving DSA problems on hackerrank and geeksforgeeks which gave me more confidence. I admit that solving them was not easy but I had google and youtube to my rescue always. By then I had also learnt the basics of html,css,Javascript which helped me in creating some more projects. Then I learnt flutter and started coding up apps .I get very happy even today when I see the apps which I have made. I can say that I can create basic to medium sized apps all by myself without taking help from others😃(obviously stackoverflow,youtube,google to be excluded). That's why I love flutter :D

Seeing the things which you have created gives you high level of satisfaction. It's not that easy to code because you will get stuck in errors many times but you just need to be consistent and google enough until you can solve the error:p Thus this teaches you to be patient enough. Once you have learnt the basics, you always feel motivated to learn more and explore on your own.You can always feel the high level of satisfaction and happiness at the end of the day after a fruitful day of coding. And that's the best thing you can ever feel . And yes, it adds a lot to your resume.

So the change in my life is that I have started living a happy life compared to the normal boring life before. The friend circle which I have developed has also changed .They motivate me to learn more and more. I feel confident enough to solve problems on my own without getting stresssed. So if you needed a sign to get out of normal life and start coding…THIS IS IT.

If you found this useful, you know what to do now. Hit that clap button and follow me to get more articles and tutorials on your feed.❤❤

StatelessWidget and StatefulWidget in Flutter.

Codeitout — Sun, 15 Nov 2020 08:06:46 +0000

A Flutter Widget extends the standard classes. The class extended determines the type of widget we are dealing with.
There are 2 classes which you will be extending 99% of the times. They are

1)StatelessWidget

The Widgets which extend StatelessWidget never change.
For eg: Text Widget(which displays text),Icon Widget(which shows the icons),their state never changes. They are also called as stateless Widgets because they don't have any state.
Example of such class:

class Codeitout extends StatelessWidget {
       Widget build(BuildContext context) {
            return new Text("Codeitout!");
       }
}

2)StatefulWidget

The Widgets which extend StatefulWidget change their state(some changes occur) whenever a user interacts with it.
For eg: TextField widget,CheckBox,etc.
When we extend StatefulWidget,we actually need to create 2 classes:
a)the stateful widget class
b)a state class

Example of it is:

class NavBar extends StatefulWidget {
  @override
  _NavBarState createState() => _NavBarState();
}

class _NavBarState extends State<NavBar> {
  int _index=0;
  final List<Widget> _page=[
    Dummy(Colors.yellow,0),
    Dummy(Colors.pink,1),
  ];
  void onTabTapped(int index) {
   setState(() {
     _index = index;
   });
 }

  @override
  Widget build(BuildContext context) {
   return Scaffold(
     body: Center(
        child: _page.elementAt(_index),
      ), // new
     bottomNavigationBar: BottomNavigationBar(
       elevation: 20,
       onTap: onTabTapped, // new
       currentIndex: _index, // new
       items: [
         new BottomNavigationBarItem(
           icon: Icon(Icons.home),
         ),
         new BottomNavigationBarItem(
           icon: Icon(Icons.mail),
         ),

       ],
     ),
   );
  }
}

Creating a LockScreen UI using Shimmer effect for text.

Codeitout — Thu, 05 Nov 2020 06:29:31 +0000

In this tutorial,We are going to design UI for a lockscreen using shimmer effect.Being an app developer you might have come across situations where you need to design a UI with amazing text effects.So that's where Shimmer effect comes into picture.

So first of all,we need to install the shimmer package by adding the following code under dependencies in pubspec.yaml file as shown below.

Now save the file and go to main.dart file and import the shimmer package using

import "package:shimmer/shimmer.dart";

In our UI we are going to display the date and time on the wallpaper and then at the end of the screen we will show "swipe to unlock".

So let's begin.

We will delete the code in MyHomePage() Class and define that class in the way we want.So we will return a Card Widget within Container which will have a child as an Stack Widget since we want to display one widget over the other (Text over Image) So in children: we will first display the image.

Now we need to add time ,date and day in between on the image.So we will use Padding Widget to have some padding and then use Text Widget to show day,date and time.Here I have directly written the date,day and time.But you can format the time,date etc. using intl package.(You can learn about how to use intl package from here.)

Now we will use shimmer effect on text "Swipe to unlock".The syntax for using it is Shimmer.fromColors() where you need to define the baseColor,highlightColor, child. So now we will wrap the text widget within the padding widget so that the text appears at bottom of screen and we will also format the text using fontSize and fontStyle as shown below.

This is the final code for main.dart

You can also watch my video tutorial if you like learning from videos:)

If you found this tutorial useful, you know what to do now. Hit that like button and follow me to get more articles and tutorials on your feed.❤❤

Participated in Hacktoberfest for the first time.

Codeitout — Tue, 03 Nov 2020 09:15:56 +0000

I had never contibuted to Hacktoberfest or any open source projects before.So when I came to know about Hacktoberfest for the first time i.e this year,I decided to participate in it.

So today I will tell you about my experience as a first time contributor in Hacktoberfest.

I started off by registering up for the event. But the main thing was to find the repositories where you could contribute or solve the issues. I found a section in the official website which was dedicated for first time contributors. There anyone could understand the process of contributing to the hacktoberfest and it's rules. Once I read it ,I started checking out the repositories where I could contribute and since I was good in flutter and few other programming languages, I found few repositories. Initially I completed 4 PRs too which is the minimum criteria for completing hacktoberfest.Those PRs were successfully merged too. I was feeling kinda happy(I dont know why) to contribute to the github repos,so I contributed to more repositories of my interest and did 9 PRs.

After completing the 4 successful PRs,I was very impatient to wait for the 14 day review period.And my PRs passed the review period too. Then finally I recieved the mail that I have completed the hacktoberfest successfully and thus now I will get the cool Hacktoberfest tee too :p

If you found this useful, you know what to do now. Hit that like button and follow me to get more posts and tutorials on your feed.❤❤

How to get started with Flutter?

Codeitout — Sun, 01 Nov 2020 04:17:52 +0000

Flutter is one of the latest tech which the developers are getting obsessed with.You will fall in love with flutter once you have learned the basics of it,just like me 😛.
So Let’s get started.

What is flutter actually?

Flutter is Google’s UI toolkit for building beautiful, natively compiled applications for mobile, web, and desktop from a single codebase.If you’re building apps in limited time with a limited budget, Flutter is definitely worth trying. It’s just as good as it seems to be, and with each new update, Google devs add more even tools for cross-platform development.Using flutter will give you lot of advantages.

. One code for 2 platforms.
. Less testing.
. Faster apps.
. Designs which everyone loves.
. Faster code writing.
. It has the hot reload feature.

Flutter apps are written using Dart programming language. Flutter requires Flutter SDK. The IDEs to be used are:
Xcode
Android Studio
VScode

You can check this link on how to set up your favourable editor. You can also check out some youtube videos on how to set these up.Once you have set them up you can start by learning Dart language.

What is Dart?

Dart is a client-optimized programming language for apps on multiple platforms. It is developed by Google and is used to build mobile, desktop, server, and web applications. Dart is an object-oriented, class-based, garbage-collected language with C-style syntax.
You can learn Dart from the following links :

A tour of the Dart

Dart tutorials

These tutorials teach you how to use the Dart language, tools, and APIs to build applications. If you want a hands-on coding experience, try a codelab.You can also use dartpad to practice Dart.You can also switch to youtube videos if you like learning from videos.

Start coding your first Flutter App

Flutter renders the app in the form of Widgets.The central idea is that you build your UI out of widgets. Widgets describe what their view should look like given their current configuration and state.UI designing involves playing with the Widgets.Flutter is very easy to learn. Flutter can be learned by watching Videos, Reading Documentations and Blogs, and practicing example.
You can learn flutter from the following tutorials.

flutter tutorial link 1

flutter tutorial link 2

flutter tutorial link 3

I learned Flutter from Udemy,the link has been provided above.
I have my own channel where I upload Flutter videos which will be quite useful.So don’t forget to check that out :p

Codeitout

There are many others sources too which you can explore by yourself.You can also join the official channels of Flutter on Github,Slack,Flutter Community on medium,etc.

Once you have learned Flutter using the basic courses,now its time to explore for more things in flutter.You can always check out the flutter documentation to learn new stuffs.Keep reading more and more articles on flutter everyday and yeahh,you need to keep on practicing these stuffs regurarly too.Code out your own projects.This will make you feel more confident.

So now you know what to do.If you found this useful, you know what to do now. Hit that like button and follow me to get more posts and tutorials on your feed.❤❤