DEV Community: Tetiana Fydorenchyk

Java Garbage Collection Types and Settings

Tetiana Fydorenchyk — Wed, 21 Aug 2019 18:46:31 +0000

Performance and price are two big considerations in application hosting that always matter. And, often, we question ourselves on how to decrease the spends, without affecting the performance of your apps at the same time. In this article, we’d like to address automatic memory management for Java applications hosted with Jelastic using garbage collection.

Let’s clarify what garbage collection is, what it does for Java applications and how it works within Jelastic PaaS.

Java Garbage Collection Overview

Garbage Collection is a form of automatic memory management. Its aim is to find data objects in memory that are no longer demanded and make their space available for reuse.

The created object uses some memory that remains allocated until there are references for the use of the object. When there are no references for an object, it is considered to be no longer required and the memory occupied by the object can be reclaimed. In such a way, you don’t pay for unused resources and can cut your costs.

We’ve tested different kinds of Garbage Collectors (GC) and defined the most appropriate ones for the Java applications hosted in our cloud, taking into consideration the automatic vertical scaling that Jelastic provides. As a result of our investigations, we adjusted the default settings of Garbage Collectors in Jelastic in order to increase the benefits for our users.

Jelastic supports the following GCs:

G1 GC (-XX:+UseG1GC) is a default GC in Jelastic PaaS. The Garbage-First (G1) is a server-style Garbage Collector for multiprocessor machines with a large amount of memory. The heap is partitioned into fixed-sized regions and G1 tracks the live data in those regions. When Garbage Collection is required, it collects from the regions with less live data first.
Shenandoah GC (-XX:+UseShenandoahGC) is a concurrent garbage collector for the JVM. Concurrent means that the GC tries to perform most of the activities in parallel without interrupting application performance. Such parallelism makes “stop-the-world” (STW) pauses extremely short that is the most required task for each GC. Another inherent advantage is an efficient work with small and large heaps with no impact on STW pauses’ length. Note that for using this GC, you need to unlock experimental options with -XX:+UnlockExperimentalVMOptions.
ZGC (-XX:+UseZGC) is low latency scalable garbage collector. Designed for use with applications that require a large heap and low latency. It uses a bunch of one generation and performs most (but not all) garbage collection in parallel with uninterrupted application work. This greatly limits the impact of garbage collection on your application response time. Note that before calling this GC type, you need to unlock experimental options with -XX:+UnlockExperimentalVMOptions.
Epsilon GC (-XX:+UseEpsilonGC) is a passive GC that handles memory allocation and doesn’t clear it when objects are no longer used. When your application exhausts the Java heap, the JVM goes down. So, EpsilonGC prolongs an application life until the memory will run out and dumps the memory, that can be useful for application memory usage debugging, as well as measuring and managing application performance. Note that before calling this GC type, you need to unlock experimental options with -XX:+UnlockExperimentalVMOptions.
Parallel
ParNew GC (-XX:+UseParNewGC) is a “stop-the-world” multithreaded Garbage Collector. Mostly it is aimed to collect the young generation objects. Since the young generation is normally small in size, the ParNew does collection very fast and does not impact your application too much. In addition, ParNew has compaction of unused RAM that enables support of automatic vertical scaling – one of the prominent Jelastic features.
Parallel GC (-XX:+UseParallelGC) is used when the parallel collection method is required over young generation only. It cannot be applied along with ConcMarkSweep GC simultaneously unlike ParNew GC.
Parallel Old GC (-XX:+UseParallelOldGC) utilizes a parallel “mark and compact” algorithm which catches all application threads and then handles labeling and subsequent compaction with multiple garbage collector threads.
ConcMarkSweep GC (-XX:+UseConcMarkSweepGC) collector is designed for applications that prefer shorter garbage collection pauses and which can afford to share processor resources with the garbage collector while the application is running. It makes sense to use such a collector when applications requirements for time garbage collection pauses are low.
Serial GC (-XX:+UseSerialGC) performs garbage collection in a single thread and has the lowest consumption of memory among all GC types but, at the same time, it makes long pauses that can lead to application performance degradation.

Default JVM Options in Jelastic PaaS

By default Jelastic PaaS uses G1 GC for JVM 8+ versions. For lower versions it employs the ParNew GC. Also, for JVM versions below 12 Jelastic attaches jelastic-gc-agent.jar which enables vertical scaling for older releases.

For JVM 12+ versions, the platform provides integrated vertical scaling to ensure G1 triggering with the following pre-set container variables:

G1PERIODIC_GC_INTERVAL=3000 Interval between garbage collection in milliseconds (15 minutes by default)
GC_SYS_LOAD_THRESHOLD_RATE=0.3 Custom multiplier to flexibly adjust the G1PeriodicGCSystemLoadThreshold value
G1PERIODIC_GC_SYS_LOAD_THRESHOLD={CPU_cores_number}*GC_SYS_LOAD_THRESHOLD_RATE Activates garbage collection, if the average one-minute system load is below the set value. This condition is ignored if set as zero.

You can always check current settings of your Java process by executing ps -ax | grep java. You will see something like this:

/usr/java/libericajdk-12.0.1/bin/java.orig -server -XX:G1PeriodicGCSystemLoadThreshold=0.6 -XX:G1PeriodicGCInterval=900k -XX:+UseStringDeduplication –

XX:+UseG1GC -Xmaxf0.3 -Xminf0.1 -Xmx1638M -Xmn30M -Xms32M -jar jelastic-helloworld-1.1.war

Also Jelastic automatically configures the following parameters:

Xmx – 80% of total available RAM in the container
Xms – 32MB
Xmn – 30MB

If JVM version is higher than 12, the platform additionally configures the following Java options:

G1PeriodicGCSystemLoadThreshold=CPU_COUNT*0.3 30% of load average based on number of CPU cores available in the container
G1PeriodicGCInterval=900k 15 minutes should pass since any previous garbage collection pause For more details, you can review the following script that manages automatic configuration of the Java options.

Customization of GC Settings in Jelastic PaaS

If you believe that customization of default settings can improve performance or memory consumption, you can tune them according to the requirements of your application. We recommend customizing these configurations only if you fully understand the impact of such changes on your application behaviour.

You can set a custom GC parameter based on your application requirements via Environment Variables (please do not mix them with Java options).

_JAVA_OPTIONS and JAVA_TOOL_OPTIONS – please read more about these options. Java options can be used for changing default GC type, for example: _JAVA_OPTIONS=”-XX:+UnlockExperimentalVMOptions -XX:+UseShenandoahGC”
GC_DEF – type of Garbage Collector, for example GC_DEF=G1GC
XMX_DEF_PERCENT – RAM percentage to be provided as XMX, for example XMX_DEF_PERCENT=80
XMX_DEF (or just XMX) – maximum size for the Java heap memory, for example if total RAM is 2048Mb the XMX_DEF=1638
XMS_DEF (or just XMS) – initial java heap size, for example XMS=32M
XMN_DEF – the size of the heap for the young generation, for example XMN=30M
G1PERIODIC_GC_INTERVAL (for openJDK 12/13 only) – a frequency of the G1 Periodic Collection in milliseconds (G1PeriodicGCInterval – 15 minutes by default); set as 0 to disable, G1PERIODIC_GC_INTERVAL=900
G1PERIODIC_GC_SYS_LOAD_THRESHOLD (for openJDK 12/13 only) – allows G1 Periodic Collection execution, if the average one-minute system load is below the set value; This condition is ignored if set as zero. By default, it is equal to the {CPU_cores_number}*{GC_SYS_LOAD_THRESHOLD_RATE}
GC_SYS_LOAD_THRESHOLD_RATE (for openJDK 12/13 only) – custom multiplier to flexibly adjust the G1PeriodicGCSystemLoadThreshold value (0.3 by default), for example G1PERIODIC_GC_SYS_LOAD_THRESHOLD_RATE=0.3
FULL_GC_AGENT_DEBUG – enables (true) or disables (false) the debug mode to track the Java GC processes in the logs, for example, FULL_GC_AGENT_DEBUG=true
FULL_GC_PERIOD – Sets the interval (in seconds) between the full GC calls; 900 by default, i.e. 15 minutes, for example FULL_GC_PERIOD=900
MAXPERMSIZE– automatically defined only for those Java containers, which run JVM version lower than 8th and with an allocated amount of RAM > 800 MiB. In all other cases (i.e. if container scaling limit is less than 7 cloudlets or it uses Java 8) this parameter is omitted. The actual value of the MaxPermSize setting is calculated based on Xmx memory amount divided by ten, but cannot be set greater than maximum of 256 MiB. For example, MAXPERMSIZE=163
XMINF_DEF– this parameter controls the minimum free space in the heap and instructs the JVM to expand the heap, if after performing garbage collection it does not have at least XMINF_DEF value of free space. For example, XMINF_DEF=0.1
XMAXF_DEF – this parameter controls how the heap is expanded and instructs the JVM to compact the heap if the amount of free space exceeds XMAXF_DEF value. For example, XMAXF_DEF=0.3

Alternatively, all these parameters can be passed to Java process via variables.conf in the container.

All of the paths to config, executable or log files can differ based on the Java server you use and can be accessed via Configuration File Manager or SSH.

Open Conf files to configure your Java server.

For Tomcat, navigate to the opt > tomcat > conf > variables.conf file.

In the opened variables.conf file you can override garbage collector default settings or even add another GC to replace the default one (G1). So if you want to use ShenandoahGC instead, simply add it to the variables.conf as stated in the example below:

     -XX:+UnlockExperimentalVMOptions

     -XX:+UseShenandoahGC

After this, only the specified garbage collector will be used while starting your Java server without taking into consideration the amount of allocated resources.

Also, you can control how JVM handles its heap memory with other JAVA options stated in this file.

As a result of properly configured options, the GC can be observed in action via the Statistics tab.

That’s it! Enjoy resource efficiency while running your Java applications in the cloud. Try it yourself with Jelastic Multi-Cloud PaaS.

Scaling Kubernetes on Application and Infrastructure Levels

Tetiana Fydorenchyk — Thu, 08 Aug 2019 09:12:18 +0000

Once an internal project from Google, Kubernetes has changed the way software development is done these days. A white steering wheel on a blue background seems to be everywhere now. Business wants to grow and pay less, DevOps want a stable platform that can run applications at scale, developers want reliable and reproducible flows to write, test and debug code. Kubernetes promises it all. Now that Jelastic offers managed Kubernetes, spinning up a cluster can’t be easier.

However, have you given some thoughts on how to get such a powerful container orchestration platform and pay just for resources that you actually need? This article will shed some light on horizontal and vertical scaling both on Jelastic (infra) and Kubernetes (application) level.

Scaling Kubernetes on Infrastructure Level

A Kubernetes cluster typically consists of a master (or a couple of them) and multiple nodes where application pods are scheduled. The maths is quite simple here: the more applications you run in your cluster, the more resources (nodes) you need. Say, you have a microservices application consisting of 3 services, each started as an individual pod which requests 1GiB of RAM. It means that you will need a 4 GiB node (K8s components and OS will require some RAM too). What if you need additional RAM in case of high load, potential memory leaks or if you deploy more services to the cluster? Correct, you either need a larger node or add an additional node to the cluster. Usually in both cases, you will pay for the exact amount of resources that come with a VM (i.e. you will pay for, say, 3GiB of RAM even if half of it is unused). That’s not the case with Jelastic though.

Vertical Scaling of Kubernetes Nodes

Let’s get back to maths. If the application roughly needs 3GiB of RAM, and there’s not much going on in the cluster, you need just one node. Sure thing, having some extra free RAM is always a good idea, so a 5GiB node makes a lot of sense.

Again, not with Jelastic. What you can do is request a 3GiB node and have a 2GiB in stash. When your application (K8s pod) starts consuming more (which is configured on K8s side too) or you simply deploy more pods (as in the chart below), those 2 extra GiB become immediately available, and you start paying for those resources only when they are used.

As a result, you can do some simple math and figure out the best cluster topology: say, 3 nodes, with 4 GiB of reserved RAM and 3GiB of dynamic resources.

Horizontal Auto-Scaling of Kubernetes Nodes

Having one huge node in a Kubernetes cluster is not a good idea since all deployments will be affected in case of an outage or any other major incident. Having several nodes in a stand-by mode is not cost efficient. Is it possible that Kubernetes adds a node when it needs it? Yes, a Kubernetes cluster in Jelastic can be configured with horizontal node auto-scaling. New nodes will be added to a cluster when RAM, CPU, I/O or Disk usage reaches certain levels. Needless to say, you get billed for additional resources only when they are used. Newly added nodes will be created according to current topology, i.e. existing vertical scaling configurations will be applied. The system will scale down as soon as resource consumption gets back to expected levels. Your Kubernetes cluster will not starve, yet you will not pay for unused resources.

Scaling Kubernetes on Application Level

Kubernetes has its own horizontal pod auto-scalers (HPA). In simple words, HPA will replicate chosen deployments based on utilization of CPU. If CPU consumption of all pods grows more than, say, 70%, HPA will schedule more pods, and when CPU consumption gets back to normal, deployment is scaled back to the original number of replicas.

Why is this cool and how does it work with automatic horizontal scaling of Kubernetes nodes? Say, you have one node with a couple of running pods. All of a sudden, a particular service in the pod starts getting a lot of requests and performing some CPU costly operations. RAM utilization does not grow, and as a result at this point there is no mechanism to scale the application which will soon become unresponsive. Kubernetes HPA will scale up pods, and an internal K8s load balancer will redirect requests to healthy pods. Those new pods will require more resources, and this is where Jelastic horizontal and vertical scaling comes into play. New pods will be either placed on the same node and utilize dynamic RAM, or a new node will be added (in case there’s not enough resources on existing ones).

Kubernetes Scaling Out

Kubernetes Scaling In

On top of that, you may set resource caps on Kubernetes pods. For example, if you know for sure that a particular service should not consume more than 1GiB, and there’s a memory leak if it does, you instruct Kubernetes to kill the pod when RAM utilization reaches 1GiB. A new pod will start automatically. This gives you control over resources your Kubernetes deployments utilize.

Living Proof with WordPress Hosted in Kubernetes

Now, let’s deploy a real-life application to a Kubernetes cluster to show all of the above mentioned scaling features. A WordPress site would be a great example. Huge Kubernetes community is one of its biggest advantages and adoption factors, so it’s really easy to find tutorials on how to deploy popular applications. Let’s go through the official WordPress tutorial (non-production deployment is chosen for simplicity of this article, and you may deploy WordPress using helm charts).

Once done, there is one more thing to do. We will need to create an Ingress bound to a WordPress service since access to running applications in Jelastic Kubernetes cluster is provided by Traefik reverse proxy.

Create a file called wp-ingress.yaml with the following content:

apiVersion: extensions/v1beta1
kind: Ingress
metadata:
labels:
app: wordpress
name: wp
annotations:
kubernetes.io/ingress.class: traefik
ingress.kubernetes.io/secure-backends: "true"
traefik.frontend.rule.type: PathPrefixStrip
spec:
rules:

http: paths:
- path: / backend: serviceName: wordpress servicePort: 80

After ingress is created, a WordPress site is available at https://.

Great, we have a running WordPress instance in a Kubernetes cluster. Let’s create horizontal pod auto-scalers (HPA) for WordPress deployment to make sure the service always responds despite high load. In your terminal, run:

kubectl autoscale deployment wordpress --cpu-percent=30 --min=1 --max=10 -n wp

Now, if WordPress pod starts utilizing >30% of CPU for all pods, autoscaler will modify deployment to add more pod replicas, so that an internal load balancer routes requests to different pods. Of course, chosen values are for demo purposes only and can be adjusted based on your needs.

Next step is to review vertical and horizontal scaling options of a Jelastic Kubernetes environment. There are two main goals here:

Pay only for actually utilized resources
Make sure Kubernetes cluster has a spare node when required

To make the demo simpler (i.e. run out of RAM faster), let’s configure the node to use up to 1.5GiB RAM and 4.8GHz CPU. Click Change Environment Topology and state Scaling Limit per node.

Configuration of an automatic horizontal scaling is the final step. Similar to vertical scaling, values are set to be promptly triggered to suit purposes of this demo. Let’s instruct Jelastic when to add and remove Kubernetes nodes. Open Settings > Auto Horizontal Scaling and Add a set of required triggers.

Let’s take a look at memory consumption. Kubernetes node (Workers) uses 8 cloudlets with 4 more cloudlets that can be dynamically added, i.e. there is some RAM to schedule a few more pods.

It’s time to put some stress on a WordPress site. There are multiple ways to initiate HTTP GET requests. We will use the simplest – wget in a while-true loop executed within WordPress pod itself (wordpress is a service name that will be resolved to an internal IP accessible from the cluster only):

while true; do wget -q -O- http://wordpress; done

A few moments later, we can observe the following data from HPA:

Kubernetes autoscaler has modified WordPress deployment to add more replicas. Expectedly, the cluster does not have enough RAM, so as Kubernetes Dashboard suggests, the remaining 5 pods cannot be scheduled:

A few pods were able to start though, as Jelastic dynamically added cloudlets (RAM&CPU) to the node. However, the remaining 2 cloudlets were not enough for at least one more pod to start.

This is where the magic begins. Since we have configured automatic node auto scaling, Jelastic is adding a new node now.

Let’s check if a new node is registered with master by running a simple command kubectl get nodes

A few moments later:

Fantastic! All pods have started which means WordPress can handle all those incoming requests that we have initiated a few minutes ago. Now, let’s abort this while true command and wait for a minute or so. Replica count gets back to 1 again which means we do not need an additional node.

Memory utilization figures suggest that the node can be deleted.

And Jelastic indeed removes it in about one minute.

What have we just witnessed? Let’s recap. Creating HPA for WordPress deployment initiates scheduling of more pod replicas to handle high load. It is up to application admin to configure triggers. Next, Jelastic dynamically allocates RAM within nodes. If you run out of RAM, a new node is added to the cluster. When resource utilization gets back to normal, HPA sets replica count back to 1, and Jelastic removes a node. When HPA scales up deployment again, Jelastic reacts accordingly. No extra RAM is in use at any time! No downtime!

With so many scaling options, both on cluster (infrastructure) and application (deployment/pod) level, a Kubernetes cluster in Jelastic becomes a smart platform that either grows or shrinks according to your application workloads. Kubernetes even checks if your application is up and running, and redeploys it if necessary with zero downtime when the application is being updated with new images, which makes continuous delivery a reality, not just a buzzword. Try out yourself at one of Jelastic public cloud service providers or request private cloud installation with Kubernetes cluster.

Magento Cloud Hosting. How to Deploy E-Commerce Platform

Tetiana Fydorenchyk — Wed, 31 Jul 2019 11:43:37 +0000

The online shopping industry is gaining popularity all over the world, so e-commerce sites are extremely demanded nowadays. We can find many options available online to build and host an e-commerce website, but Magento cloud hosting is considered to be one of the most robust, flexible and scalable open source e-commerce platforms on the market.

Magento includes a wide range of tools to build and manage an online store. It is well known in the developer community for its use of object-oriented programming (OOP) and especially for its use of the Entity–attribute–value (EAV) model. This allows Magento to be enough versatile at storing a large range of data.

Magento can be automatically deployed and hosted in the cloud with the help of Jelastic PaaS. There are two installation packages with different web-servers used in the topology – NGINX or LiteSpeed. It’s pretty easy to get them up and running, just follow a few simple steps below.

1.Enter the Jelastic dashboard, find the Magento Standalone package in the Marketplace and press Install.

Alternatively, you can import the Magento manifest with the needed topology:

2.In the opened popup dialog, specify the name of Environment and choose the preferred Region.

If the installation of Magento on LiteSpeed Web Server is available on the chosen service provider, there will be the checkbox Install LiteSpeed High-Performance Web Server. And if you prefer NGINX-based installation, just untick this option.

Click on the Install button.

3.Wait a few minutes till the installation is completed.

4.Follow the Magento environment link to open Magento welcome page.

5.Use credentials from the successful installation message or received email notification to access the admin panel and start managing your e-commerce platform.

Everything is set up and ready! Magento is not only user-friendly but also beneficial for merchants as they can manage multiple stores within a single admin panel. It is a perfect shopping cart that meets the requirements of today’s competitive business. Try it out yourself at one of public Jelastic PaaS service providers.

We hope you found this post useful. If you have any experience with scalable Magento hosting please share your viewpoint in the comment section below.

What is PaaS? Platform-as-a-Service Types Explained

Tetiana Fydorenchyk — Tue, 16 Jul 2019 15:06:44 +0000

There’s no single PaaS for everyone. Even the definition of this cloud computing service model is quite vague today. Commonly, it is described as a cloud hosting platform with a set of deployment and scaling automation, application management and DevOps tools that can be run on shared infrastructure or on-premise. Also, PaaS eliminates the complexity of building and maintaining the underlying infrastructure. Let’s see how PaaSes differ through the prism of their use cases and supported technologies, and how to determine which one will meet specific needs.

Let’s take a closer look and analyze Platform-as-a-Service solutions from several angles:

purpose
abstraction
functionality

Purpose-Driven Categories of PaaS

Today’s PaaS vendors bring innovative approaches to application development and their hosting in the cloud by providing various tools built specifically for a particular service. Considering use cases hosted within Platform-as-a-Service solutions, we can break down PaaS offerings into three broad categories: general-purpose, emerging and specialized.

General-purpose platforms are widely chosen for traditional stateful web and N-tier applications thus offering “lift and shift” approach for migrating existing workloads. In addition, this PaaS type is timely expanding with modern technologies that meet the needs of stateless cloud-native applications and microservices, and help to develop new scalable and fault tolerant services.

Emerging PaaSes bring new approaches to the market. They are commonly used for development of independent services based on emerging technologies such as serverless, distributed event processing, machine learning frameworks, and others.

Specialized platforms focus on narrow niche use cases but with high demand on the market. Such products have proved to be effective in different directions including e-commerce, big data processing and business applications. Often, these platforms are transformed into subcategories such as CMSaaS, eCommerce-as-a-Service, DBaaS, BPaaS, etc.

Abstraction Levels of PaaS

PaaS solutions are increasingly focused on full-cycle automation of application deployment and delivery processes. This helps developers to focus more on application design and its implementation instead of time consuming routine or complex tasks, such as server and storage configuration, application and infrastructure security, auto scaling, load balancing, and so on.

There can be considered three levels of PaaS abstraction: low, mid and high.

Low abstraction level platforms, especially Container-as-a-Service (CaaS) solutions, provide DevOps teams with deep access to infrastructure configurations, as well as granular control over containers and various platform services (e.g. service discovery, logging, security and load balancing).

Mid abstraction has the primary goal to let developers code without headache of infrastructure configuration and management. These platforms give developers access to a wide range of integrated and optimized development frameworks, middleware software stacks, APIs and services completely abstracted from CaaS and IaaS deployment properties. Very often such PaaS solutions provide automatic scaling, load balancing, high-availability, backup, disaster recovery, CI/CD, and other application lifecycle management features out-of-box.

High abstraction solutions are aimed to abstract even coding process to increase the application delivery speed and reduce the cost of innovations. In this case, developers get the entire technology stack but with full abstraction of application, platform and infrastructure levels. It helps to boost performance of team members and reduce requirements to their technical skills.

Functionality and Services Integrated to PaaS

In order to automate the application lifecycle management processes, PaaS represents a collection of configuration tools, continuous integration and delivery services, connectors and plugins to popular IDEs.

We can highlight three main approaches in this category:

Deployment Automation

PaaSes concentrated on providing tools and services for deployment automation offering “one-click” or “git-push” functionality, as well as pre-configured environments for code building, continuous integration and project management tools.

Infrastructure Management

Cloud platforms focused on infrastructure configuration, health and availability monitoring, security and performance checks, cost optimization, as well as offering automation and services related to upgrades and maintenance.

Container Orchestration

Container management platforms offering functionality to solve challenges of containerization process and implementation of cloud-native approach for existing or new applications. Many PaaS vendors now offer CaaS as an integral component of their products, further reinforcing the view of PaaS as a collection of different DevOps tools.

Make the PaaS Choice

There are several important points that should be considered before selecting a PaaS for a company.

Above all, analyze if the chosen PaaS supports the technologies, middleware stacks and programming languages used by development teams today or planned to be used in the future.

The next point to consider is vendor and data lock-in. Despite benefits some PaaSes offer, the access to the application itself and used middleware stacks can be very limited due to proprietary design of tools and services. This aspect affects the application portability. So migration to any other cloud or implementing multi-cloud interoperability approach becomes a real challenge. Same issue applies to the stored data. Ensure that PaaS provider allows to import/export data easily without any lock-in.

They say there’s no “one-size-fits-all” technology for every project but in modern cloud world with all integrations and interconnections available the borders between specific products are blurring. Jelastic Multi-Cloud PaaS provides the required freedom of choice when it comes to cloud services, DevOps tools and container management systems. Give it a try to immensely improve development productivity and application delivery processes with minimal investment.