<?xml version="1.0" encoding="UTF-8"?>
<rss version="2.0" xmlns:atom="http://www.w3.org/2005/Atom" xmlns:dc="http://purl.org/dc/elements/1.1/">
  <channel>
    <title>DEV Community: Sergii</title>
    <description>The latest articles on DEV Community by Sergii (@sergiidemianchuk).</description>
    <link>https://dev.to/sergiidemianchuk</link>
    <image>
      <url>https://media2.dev.to/dynamic/image/width=90,height=90,fit=cover,gravity=auto,format=auto/https:%2F%2Fdev-to-uploads.s3.amazonaws.com%2Fuploads%2Fuser%2Fprofile_image%2F829156%2F06e68bf3-5289-41f3-bf41-2d8fd8efd552.jpeg</url>
      <title>DEV Community: Sergii</title>
      <link>https://dev.to/sergiidemianchuk</link>
    </image>
    <atom:link rel="self" type="application/rss+xml" href="https://dev.to/feed/sergiidemianchuk"/>
    <language>en</language>
    <item>
      <title>PHP REST API Cybersecurity</title>
      <dc:creator>Sergii</dc:creator>
      <pubDate>Sat, 17 May 2025 10:21:04 +0000</pubDate>
      <link>https://dev.to/sergiidemianchuk/php-rest-api-cybersecurity-41j2</link>
      <guid>https://dev.to/sergiidemianchuk/php-rest-api-cybersecurity-41j2</guid>
      <description>&lt;p&gt;Hi, current article is a short representation of my course: “PHP REST API Cybersecurity”.&lt;/p&gt;

&lt;p&gt;In an increasingly connected world, cybersecurity is no longer just an IT concern – it’s a critical issue that affects businesses, individuals, and entire nations. As the global landscape becomes more interconnected, and tensions between nations rise, cyberattacks are becoming more sophisticated and frequent. It’s not just about protecting against hackers; it’s about understanding how vulnerabilities in systems can be exploited, and how we can defend against them.&lt;/p&gt;

&lt;p&gt;&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.amazonaws.com%2Fuploads%2Farticles%2Fhxbel5lyt49a5f84g09l.jpeg" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.amazonaws.com%2Fuploads%2Farticles%2Fhxbel5lyt49a5f84g09l.jpeg" alt="Image description" width="800" height="450"&gt;&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;While many courses, articles, and conferences focus on ethical hacking and penetration testing, there’s a glaring gap in defensive security. Specifically, how do we protect the applications we build from these attacks? That’s why I’ve created this course – PHP REST API Security – to empower developers to understand both the offensive and defensive aspects of web application security, especially in the context of PHP and REST APIs.&lt;/p&gt;

&lt;p&gt;&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.amazonaws.com%2Fuploads%2Farticles%2Fydy6i2qu0boomo2r6jeh.jpeg" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.amazonaws.com%2Fuploads%2Farticles%2Fydy6i2qu0boomo2r6jeh.jpeg" alt="Image description" width="800" height="450"&gt;&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;In my 20 years of experience in cybersecurity, I’ve often wondered why there are so many courses on how to hack into systems, but so few that focus on how to protect against those attacks. The defensive side of security is significantly more complex. It’s not just about knowing how to break things—it’s about understanding the intricate details of various technologies, frameworks, and languages, like PHP, and how they work together in an application.&lt;/p&gt;

&lt;p&gt;To be a good defender, you need to have a deep understanding of the technology you’re working with. You need to know the ins and outs of programming languages, operating systems, and infrastructure architecture. It’s a multi-faceted, more difficult approach that requires much more effort to master, which is why there are fewer experts in this area.&lt;/p&gt;

&lt;p&gt;In this course, I’ve condensed over almost 2 decades of experience into 5 hours of practical learning, designed to help you understand the top vulnerabilities in PHP REST APIs and how to mitigate them effectively.&lt;/p&gt;

&lt;p&gt;First several sections focuses on indirect security issues that are essential for every developer to understand. API documentation, error handling, and debugging are all areas that are often overlooked but can have a huge impact on the overall security of your system.&lt;/p&gt;

&lt;p&gt;&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.amazonaws.com%2Fuploads%2Farticles%2Ftn13f5xiog7tw5i2xf7m.jpeg" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.amazonaws.com%2Fuploads%2Farticles%2Ftn13f5xiog7tw5i2xf7m.jpeg" alt="Image description" width="800" height="450"&gt;&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;At next step the most popular formats for APIs would be discussed, with a quick dive into why JSON is generally the safest and why XML is far from a good choice. You will learn in details on practice about Denial of service XML attacks such as Quadratic Blowup Attack and Billion Laughs Attack, smoothly going with XML External Entity attacks and finishing with Remote Code Execution (RCE).&lt;/p&gt;

&lt;p&gt;&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.amazonaws.com%2Fuploads%2Farticles%2F7b6ph9ng2lfgefsdkbbw.jpeg" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.amazonaws.com%2Fuploads%2Farticles%2F7b6ph9ng2lfgefsdkbbw.jpeg" alt="Image description" width="800" height="450"&gt;&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;Then I am passing over the most popular vulnerabilities and explore how to mitigate them with hands-on examples. The course covers such topics as:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;XSS attacks – reflected, stores and file-based types&lt;/li&gt;
&lt;li&gt;CSRF&lt;/li&gt;
&lt;li&gt;SQL injection – union, blind content and time based vectors&lt;/li&gt;
&lt;li&gt;Template injections (SSTI)&lt;/li&gt;
&lt;li&gt;Path Traversal&lt;/li&gt;
&lt;li&gt;Incredibly tricky theme of PHP deserialization and how it can be used as vulnerability&lt;/li&gt;
&lt;li&gt;Command Injection – here I’ll reveal a zero-day case from my recent work. And explain why red teams often have the upper hand over blue teams, and why it’s impossible to be 100% safe – even with perfect security practices.&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.amazonaws.com%2Fuploads%2Farticles%2F4iq5tn1qtoc3mqvngm0g.jpeg" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.amazonaws.com%2Fuploads%2Farticles%2F4iq5tn1qtoc3mqvngm0g.jpeg" alt="Image description" width="800" height="450"&gt;&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;Every concept is paired with practical coding exercises. For each vulnerability, we’ll recreate attacks, so you can feel them firsthand, then apply the best practices to defend against them. That is far from all, as course also icludes such essential cybersecurity issues as:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;Rate limiting&lt;/li&gt;
&lt;li&gt;Server-side request forgery (SSRF)&lt;/li&gt;
&lt;li&gt;Hidden files&lt;/li&gt;
&lt;li&gt;Webhooks&lt;/li&gt;
&lt;li&gt;Cross-Origin Resource Sharing (CORS)&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.amazonaws.com%2Fuploads%2Farticles%2Ftfix53psdwfilx771tla.jpeg" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.amazonaws.com%2Fuploads%2Farticles%2Ftfix53psdwfilx771tla.jpeg" alt="Image description" width="800" height="450"&gt;&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;There are also detailed sections about authentication and authorization problems, special focus is devoted to the JSON Web Token (JWT) vulnerabilities.&lt;/p&gt;

&lt;p&gt;&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.amazonaws.com%2Fuploads%2Farticles%2F788qyuoug8oisj4lil6i.jpeg" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.amazonaws.com%2Fuploads%2Farticles%2F788qyuoug8oisj4lil6i.jpeg" alt="Image description" width="800" height="450"&gt;&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;By the end of this course, you’ll have a solid understanding of how to secure PHP REST APIs from top to bottom. Whether you’re a beginner or an experienced developer who wants to fill in the gaps, this course is designed to give you a complete picture.&lt;/p&gt;

&lt;p&gt;If you’re ready to step up your security skills and become part of the blue team, click next link: &lt;a href="https://www.udemy.com/course/php-rest-api-cybersecurity/?referralCode=AD5370BF845F429998D5" rel="noopener noreferrer"&gt;Enroll at PHP REST API security course&lt;/a&gt;. At my &lt;a href="https://sergiiblog.com/php-rest-api-cybersecurity/" rel="noopener noreferrer"&gt;blog&lt;/a&gt; you may also find coupon for discount.&lt;/p&gt;

&lt;p&gt;Best regards&lt;/p&gt;

</description>
    </item>
    <item>
      <title>AWS OPENSEARCH BENCHMARKS</title>
      <dc:creator>Sergii</dc:creator>
      <pubDate>Sun, 14 Jan 2024 07:24:02 +0000</pubDate>
      <link>https://dev.to/sergiidemianchuk/aws-opensearch-benchmarks-1ab2</link>
      <guid>https://dev.to/sergiidemianchuk/aws-opensearch-benchmarks-1ab2</guid>
      <description>&lt;p&gt;Hi everybody. &lt;br&gt;
Recently, 04.12.2023-05.12.2023. I have a pleasant occasion to perform AWS OpenSearch service performance benchmarks. The reason for that has a rather long story, but I will try to shorten it as much as possible. There is one application that uses Elasticsearch for performing different search operations. I will call it “Search App” below in text. Current application was migrated to AWS Cloud almost 2 years ago. To not complicate the situation, at 1st migration step it was decided to move Search App from 3 “on-premise” servers  to the 3 EC2 instances preserving existing docker swarm architecture, with only some small adjustments. The simplified architecture scheme took a next view:&lt;/p&gt;

&lt;p&gt;&lt;a href="https://media.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.amazonaws.com%2Fuploads%2Farticles%2Fdy5zxau18swteu9642o4.png" class="article-body-image-wrapper"&gt;&lt;img src="https://media.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.amazonaws.com%2Fuploads%2Farticles%2Fdy5zxau18swteu9642o4.png" alt="AWS initial architecture"&gt;&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;And the 1st step was successfully done. The next migration steps were as listed below:&lt;/p&gt;

&lt;ol&gt;
&lt;li&gt;Replace Elasticsearch cluster with AWS OpenSearch service&lt;/li&gt;
&lt;li&gt;Move Search App from EC2 swarm cluster to the AWS Fargate&lt;/li&gt;
&lt;/ol&gt;

&lt;p&gt;Our final goal was to get a scalable and serverless solution. Sounds reasonable, doesn’t it? &lt;/p&gt;

&lt;p&gt;So, the 2d step was performed at the next iteration. As existing dockerized 3 node Elasticsearch cluster was limited at docker level to the 6GB RAM with JVM set to the 4GB, the solution was taken to use simplified 2 node OpenSearch cluster with m6g.large.search instance type (2 CPU, 8 GB RAM) to provide initial tests (yes, it was not HA, but at least less expensive). And in 2 weeks all was done – infrastructure with terraform, recreating index mappings, indexing tests, application unit and integration tests, and some other preparation – all problems were solved. So, we switched Search App at OpenSearch cluster.&lt;/p&gt;

&lt;p&gt;&lt;a href="https://media.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.amazonaws.com%2Fuploads%2Farticles%2Fjrn9u94joq0csxcvrz67.png" class="article-body-image-wrapper"&gt;&lt;img src="https://media.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.amazonaws.com%2Fuploads%2Farticles%2Fjrn9u94joq0csxcvrz67.png" alt="AWS migration architecture"&gt;&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;But our happiness didn’t last long. In several hours we had to switch back to the existing Elasticsearch cluster, which we preserved “just in case” ;) &lt;/p&gt;

&lt;p&gt;So, what has happened? The deal is that we did not test one functionality. Here is a short description of how it works. Users can choose/set different options at their account for getting daily reports, including concrete hour.  There is a cron at the beginning of every hour, which gathers users, whose report should be sent, and pushes according user’s ID at the SQS queue. Workers take user IDs from queues, create reports using data from Elasticsearch and send according info at user’s email. Here is simplified diagram for the whole process image:&lt;/p&gt;

&lt;p&gt;&lt;a href="https://media.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.amazonaws.com%2Fuploads%2Farticles%2Fhw8sv5bi5rbg36vt9zn6.png" class="article-body-image-wrapper"&gt;&lt;img src="https://media.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.amazonaws.com%2Fuploads%2Farticles%2Fhw8sv5bi5rbg36vt9zn6.png" alt="Cron scheme"&gt;&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;Some hours are much more popular than others. As a result, in peaks over then 10K users have to get reports. The search query by itself is rather complicated – it includes different advanced filters, including geolocation operations, and term’s aggregation based on filtered results.  Moreover, as users’ settings are extremely different, the cardinality of search requests is very high – so cache is not helpful in the current case. That creates non typical overload peaks at Elasticsearch, but the existing cluster, despite increase in memory, CPU utilization and search latency, was able to deal with that without any bigger problems. But not OpenSearch. It appeared to be not resistant to the short peaks and refused to process queries with sending 429 errors – Too many requests. If you are interested in details – please visit my course ”&lt;a href="https://www.udemy.com/course/elasticsearch-as-you-have-never-known-it-before/?referralCode=1B10488521797E430D22" rel="noopener noreferrer"&gt;AWS devops: Elasticsearch at AWS using terraform and ansible&lt;/a&gt;”. I will not reveal details here, the most essential is the final result – the decision “refuse from migration at OpenSearch” was taken. Yep, unfortunately. &lt;/p&gt;

&lt;p&gt;But almost 2 years have passed, all infrastructure has been moved to AWS Cloud. The company, owner of Search App, grew up and became a serious player. As a result AWS appeared with questions about how they can help and what problems the company would like to resolve – question of OpenSearch migration was opened again. Here, I want to say “BIG THANK YOU” to the people (Robert, Julia, Francisco), who provided me useful instructions and AWS credits for perfoming additional tests. I will not put their surnames here, though I hope they recognize themselves during the reading of the current article.&lt;br&gt;
So, my plan was the next. Deploy OpenSearch cluster using different instances type and:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;gather performance metrics using  OpenSearch Benchmark utility&lt;/li&gt;
&lt;li&gt;imitate sending reports overload at Elasticsearch/OpenSearch clusters and test it’s behaviour&lt;/li&gt;
&lt;li&gt;compare AWS OpenSearch clusters with existing Elasticsearch EC2 3 node cluster – docker memory limit 6Gb, jvm – 4GB, t3.large (2 CPU, 8 GB)&lt;/li&gt;
&lt;li&gt;compare AWS OpenSearch clusters with scaled up Elasticsearch EC2 3 node cluster – docker memory limit 14Gb, jvm – 12GB, t3.xlarge (4 CPU, 16 GB)&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;OpenSearch Benchmark utility appeared to be easy to use. The most problematic was the part related with imitating real overloading while generating reports. I had to take 10K users (anonimize their all personal data but preserve their settings), modify cron code to ignore time preferences and modify workers code – to send search requests and generate reports without sending them to email. Finally,  I indexed Elasticsearch with real production data and created an Elasticsearch snapshot from that at S3 to have the ability to dump it at different Opensearch clusters in a fast and convenient way. &lt;/p&gt;

&lt;p&gt;Results appeared to be rather interesting. You may read about here at my blog: "&lt;a href="https://sergiiblog.com/aws-opensearch-benchmarks-part-2/" rel="noopener noreferrer"&gt;AWS OpenSearch benchmarks&lt;/a&gt;"&lt;/p&gt;

&lt;p&gt;Have a pleasant reading&lt;/p&gt;

</description>
    </item>
    <item>
      <title>AWS OPENSEARCH BENCHMARKS</title>
      <dc:creator>Sergii</dc:creator>
      <pubDate>Sun, 14 Jan 2024 07:24:02 +0000</pubDate>
      <link>https://dev.to/sergiidemianchuk/aws-opensearch-benchmarks-3fj7</link>
      <guid>https://dev.to/sergiidemianchuk/aws-opensearch-benchmarks-3fj7</guid>
      <description>&lt;p&gt;Hi everybody. &lt;br&gt;
Recently, 04.12.2023-05.12.2023. I have a pleasant occasion to perform AWS OpenSearch service performance benchmarks. The reason for that has a rather long story, but I will try to shorten it as much as possible. There is one application that uses Elasticsearch for performing different search operations. I will call it “Search App” below in text. Current application was migrated to AWS Cloud almost 2 years ago. To not complicate the situation, at 1st migration step it was decided to move Search App from 3 “on-premise” servers  to the 3 EC2 instances preserving existing docker swarm architecture, with only some small adjustments. The simplified architecture scheme took a next view:&lt;/p&gt;

&lt;p&gt;&lt;a href="https://media.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.amazonaws.com%2Fuploads%2Farticles%2Fdy5zxau18swteu9642o4.png" class="article-body-image-wrapper"&gt;&lt;img src="https://media.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.amazonaws.com%2Fuploads%2Farticles%2Fdy5zxau18swteu9642o4.png" alt="AWS initial architecture"&gt;&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;And the 1st step was successfully done. The next migration steps were as listed below:&lt;/p&gt;

&lt;ol&gt;
&lt;li&gt;Replace Elasticsearch cluster with AWS OpenSearch service&lt;/li&gt;
&lt;li&gt;Move Search App from EC2 swarm cluster to the AWS Fargate&lt;/li&gt;
&lt;/ol&gt;

&lt;p&gt;Our final goal was to get a scalable and serverless solution. Sounds reasonable, doesn’t it? &lt;/p&gt;

&lt;p&gt;So, the 2d step was performed at the next iteration. As existing dockerized 3 node Elasticsearch cluster was limited at docker level to the 6GB RAM with JVM set to the 4GB, the solution was taken to use simplified 2 node OpenSearch cluster with m6g.large.search instance type (2 CPU, 8 GB RAM) to provide initial tests (yes, it was not HA, but at least less expensive). And in 2 weeks all was done – infrastructure with terraform, recreating index mappings, indexing tests, application unit and integration tests, and some other preparation – all problems were solved. So, we switched Search App at OpenSearch cluster.&lt;/p&gt;

&lt;p&gt;&lt;a href="https://media.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.amazonaws.com%2Fuploads%2Farticles%2Fjrn9u94joq0csxcvrz67.png" class="article-body-image-wrapper"&gt;&lt;img src="https://media.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.amazonaws.com%2Fuploads%2Farticles%2Fjrn9u94joq0csxcvrz67.png" alt="AWS migration architecture"&gt;&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;But our happiness didn’t last long. In several hours we had to switch back to the existing Elasticsearch cluster, which we preserved “just in case” ;) &lt;/p&gt;

&lt;p&gt;So, what has happened? The deal is that we did not test one functionality. Here is a short description of how it works. Users can choose/set different options at their account for getting daily reports, including concrete hour.  There is a cron at the beginning of every hour, which gathers users, whose report should be sent, and pushes according user’s ID at the SQS queue. Workers take user IDs from queues, create reports using data from Elasticsearch and send according info at user’s email. Here is simplified diagram for the whole process image:&lt;/p&gt;

&lt;p&gt;&lt;a href="https://media.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.amazonaws.com%2Fuploads%2Farticles%2Fhw8sv5bi5rbg36vt9zn6.png" class="article-body-image-wrapper"&gt;&lt;img src="https://media.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.amazonaws.com%2Fuploads%2Farticles%2Fhw8sv5bi5rbg36vt9zn6.png" alt="Cron scheme"&gt;&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;Some hours are much more popular than others. As a result, in peaks over then 10K users have to get reports. The search query by itself is rather complicated – it includes different advanced filters, including geolocation operations, and term’s aggregation based on filtered results.  Moreover, as users’ settings are extremely different, the cardinality of search requests is very high – so cache is not helpful in the current case. That creates non typical overload peaks at Elasticsearch, but the existing cluster, despite increase in memory, CPU utilization and search latency, was able to deal with that without any bigger problems. But not OpenSearch. It appeared to be not resistant to the short peaks and refused to process queries with sending 429 errors – Too many requests. If you are interested in details – please visit my course ”&lt;a href="https://www.udemy.com/course/elasticsearch-as-you-have-never-known-it-before/?referralCode=1B10488521797E430D22" rel="noopener noreferrer"&gt;AWS devops: Elasticsearch at AWS using terraform and ansible&lt;/a&gt;”. I will not reveal details here, the most essential is the final result – the decision “refuse from migration at OpenSearch” was taken. Yep, unfortunately. &lt;/p&gt;

&lt;p&gt;But almost 2 years have passed, all infrastructure has been moved to AWS Cloud. The company, owner of Search App, grew up and became a serious player. As a result AWS appeared with questions about how they can help and what problems the company would like to resolve – question of OpenSearch migration was opened again. Here, I want to say “BIG THANK YOU” to the people (Robert, Julia, Francisco), who provided me useful instructions and AWS credits for perfoming additional tests. I will not put their surnames here, though I hope they recognize themselves during the reading of the current article.&lt;br&gt;
So, my plan was the next. Deploy OpenSearch cluster using different instances type and:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;gather performance metrics using  OpenSearch Benchmark utility&lt;/li&gt;
&lt;li&gt;imitate sending reports overload at Elasticsearch/OpenSearch clusters and test it’s behaviour&lt;/li&gt;
&lt;li&gt;compare AWS OpenSearch clusters with existing Elasticsearch EC2 3 node cluster – docker memory limit 6Gb, jvm – 4GB, t3.large (2 CPU, 8 GB)&lt;/li&gt;
&lt;li&gt;compare AWS OpenSearch clusters with scaled up Elasticsearch EC2 3 node cluster – docker memory limit 14Gb, jvm – 12GB, t3.xlarge (4 CPU, 16 GB)&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;OpenSearch Benchmark utility appeared to be easy to use. The most problematic was the part related with imitating real overloading while generating reports. I had to take 10K users (anonimize their all personal data but preserve their settings), modify cron code to ignore time preferences and modify workers code – to send search requests and generate reports without sending them to email. Finally,  I indexed Elasticsearch with real production data and created an Elasticsearch snapshot from that at S3 to have the ability to dump it at different Opensearch clusters in a fast and convenient way. &lt;/p&gt;

&lt;p&gt;Results appeared to be rather interesting. You may read about here at my blog: "&lt;a href="https://sergiiblog.com/aws-opensearch-benchmarks-part-2/" rel="noopener noreferrer"&gt;AWS OpenSearch benchmarks&lt;/a&gt;"&lt;/p&gt;

&lt;p&gt;Have a pleasant reading&lt;/p&gt;

</description>
    </item>
    <item>
      <title>How to deploy HA Elasticsearch cluster using AWS ECS and terraform</title>
      <dc:creator>Sergii</dc:creator>
      <pubDate>Fri, 21 Jul 2023 09:02:00 +0000</pubDate>
      <link>https://dev.to/sergiidemianchuk/how-to-deploy-ha-elasticsearch-cluster-using-aws-ecs-and-terraform-1j0i</link>
      <guid>https://dev.to/sergiidemianchuk/how-to-deploy-ha-elasticsearch-cluster-using-aws-ecs-and-terraform-1j0i</guid>
      <description>&lt;p&gt;Hi, Elasticsearch fans&lt;/p&gt;

&lt;p&gt;If you are interested at "How to deploy HA Elasticsearch cluster using AWS ECS and terraform", then welcome to series of articles related to it:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;&lt;a href="https://sergiiblog.com/how-to-deploy-ha-elasticsearch-cluster-at-aws-ecs-using-terraform/"&gt;HOW TO DEPLOY HA ELASTICSEARCH CLUSTER AT AWS ECS USING TERRAFORM - PART 1&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;&lt;a href="https://sergiiblog.com/deploying-elasticsearch-cluster-at-aws-ecs-elasticsearch-docker-file-and-aws-ecr/"&gt;DEPLOYING ELASTICSEARCH CLUSTER AT AWS ECS – ELASTICSEARCH DOCKER FILE AND AWS ECR -PART 2&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;&lt;a href="https://sergiiblog.com/deploying-elasticsearch-cluster-at-aws-ecs-ecs-iam-profile-terraform-module/"&gt;DEPLOYING ELASTICSEARCH CLUSTER AT AWS ECS – ECS IAM PROFILE TERRAFORM MODULE - PART 3&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;&lt;a href="https://sergiiblog.com/deploying-elasticsearch-cluster-at-aws-ecs-ecs-ec2-terraform-module/"&gt;DEPLOYING ELASTICSEARCH CLUSTER AT AWS ECS – ECS-EC2 TERRAFORM MODULE - PART 4&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;&lt;a href="https://sergiiblog.com/deploying-elasticsearch-cluster-at-aws-ecs-ecs-cluster-terraform-module/"&gt;DEPLOYING ELASTICSEARCH CLUSTER AT AWS ECS – ECS-CLUSTER TERRAFORM MODULE - PART 5&lt;/a&gt;&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;Have a pleasant reading :)&lt;/p&gt;

</description>
    </item>
    <item>
      <title>How to work with Elasticsearch 8 using Spring Boot 3 and Spring Data Elasticsearch 5.x</title>
      <dc:creator>Sergii</dc:creator>
      <pubDate>Fri, 26 May 2023 15:05:34 +0000</pubDate>
      <link>https://dev.to/sergiidemianchuk/how-to-work-with-elasticsearch-8-using-spring-boot-3-and-spring-data-elasticsearch-5x-ahd</link>
      <guid>https://dev.to/sergiidemianchuk/how-to-work-with-elasticsearch-8-using-spring-boot-3-and-spring-data-elasticsearch-5x-ahd</guid>
      <description>&lt;p&gt;Hi, Java-Spring Boot-Elasticsearch fans&lt;/p&gt;

&lt;p&gt;Despite that Elasticsearch was upgraded from 7th to 8th version almost 2 years ago, Spring Boot and Elasticsearch package was adjusted to it relatively recently. When I had to upgrade my project to Elasticsearch 8 version with using spring boot 3 and last spring data Elasticsearch version - I've met a lot of difficulties. Mostly because of 2 main reasons:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;&lt;p&gt;too many changes appeared at how Elasticsearch queries are built since Spring Data Elasticsearch 5.x version&lt;/p&gt;&lt;/li&gt;
&lt;li&gt;&lt;p&gt;lack of good examples at internet, which impressed me a lot&lt;/p&gt;&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;As result I decide to share here some examples, which can be helpful for somebody at his upgrade path&lt;/p&gt;

&lt;p&gt;1.Example with range query&lt;/p&gt;

&lt;p&gt;Here how it was at previous version Spring Data Elasticsearch versions (3d and 4th):&lt;/p&gt;

&lt;p&gt;&lt;a href="https://res.cloudinary.com/practicaldev/image/fetch/s--ZxZeDY2b--/c_limit%2Cf_auto%2Cfl_progressive%2Cq_auto%2Cw_800/https://dev-to-uploads.s3.amazonaws.com/uploads/articles/vsn18ok2p4c7kf0gne78.png" class="article-body-image-wrapper"&gt;&lt;img src="https://res.cloudinary.com/practicaldev/image/fetch/s--ZxZeDY2b--/c_limit%2Cf_auto%2Cfl_progressive%2Cq_auto%2Cw_800/https://dev-to-uploads.s3.amazonaws.com/uploads/articles/vsn18ok2p4c7kf0gne78.png" alt="Image description" width="800" height="304"&gt;&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;Here how it looks since Spring Data Elasticsearch 5.x :&lt;/p&gt;

&lt;p&gt;&lt;a href="https://res.cloudinary.com/practicaldev/image/fetch/s--Riy4Zd1t--/c_limit%2Cf_auto%2Cfl_progressive%2Cq_auto%2Cw_800/https://dev-to-uploads.s3.amazonaws.com/uploads/articles/iqml5ou22guzklg5sz3p.png" class="article-body-image-wrapper"&gt;&lt;img src="https://res.cloudinary.com/practicaldev/image/fetch/s--Riy4Zd1t--/c_limit%2Cf_auto%2Cfl_progressive%2Cq_auto%2Cw_800/https://dev-to-uploads.s3.amazonaws.com/uploads/articles/iqml5ou22guzklg5sz3p.png" alt="Image description" width="711" height="314"&gt;&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;2.Example with bool query to create OR condition with match queries&lt;/p&gt;

&lt;p&gt;Here how it was at previous version Spring Data Elasticsearch versions (3d and 4th):&lt;/p&gt;

&lt;p&gt;&lt;a href="https://res.cloudinary.com/practicaldev/image/fetch/s--rl4Hmp0M--/c_limit%2Cf_auto%2Cfl_progressive%2Cq_auto%2Cw_800/https://dev-to-uploads.s3.amazonaws.com/uploads/articles/emrbhr9cluwfyk6yute3.png" class="article-body-image-wrapper"&gt;&lt;img src="https://res.cloudinary.com/practicaldev/image/fetch/s--rl4Hmp0M--/c_limit%2Cf_auto%2Cfl_progressive%2Cq_auto%2Cw_800/https://dev-to-uploads.s3.amazonaws.com/uploads/articles/emrbhr9cluwfyk6yute3.png" alt="Image description" width="758" height="502"&gt;&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;Here how it looks since Spring Data Elasticsearch 5.x :&lt;/p&gt;

&lt;p&gt;&lt;a href="https://res.cloudinary.com/practicaldev/image/fetch/s--wwg4sM_c--/c_limit%2Cf_auto%2Cfl_progressive%2Cq_auto%2Cw_800/https://dev-to-uploads.s3.amazonaws.com/uploads/articles/jhb2cxpbxjw6r00ebzad.png" class="article-body-image-wrapper"&gt;&lt;img src="https://res.cloudinary.com/practicaldev/image/fetch/s--wwg4sM_c--/c_limit%2Cf_auto%2Cfl_progressive%2Cq_auto%2Cw_800/https://dev-to-uploads.s3.amazonaws.com/uploads/articles/jhb2cxpbxjw6r00ebzad.png" alt="Image description" width="767" height="523"&gt;&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;The main thing that was changed is the approach of getting builders. Previously we had universal QueryBuilders class from which we got builders for different types of Elasticsearch queries. From 5x version builders are moved inside Elasticsearch “query” classes. If you are interested at more examples, please visit according article at my blog: "&lt;a href="https://sergiiblog.com/how-to-work-with-elasticsearch-8-using-spring-boot-3-x-and-spring-data-elasticsearch-5-x/"&gt;How to work with Elasticsearch 8 using Spring Boot 3&lt;/a&gt;"&lt;/p&gt;

</description>
    </item>
    <item>
      <title>HOW TO DEPLOY A HIGH AVAILABLE CLUSTER AT AWS OPENSEARCH USING TERRAFORM</title>
      <dc:creator>Sergii</dc:creator>
      <pubDate>Fri, 21 Apr 2023 09:26:59 +0000</pubDate>
      <link>https://dev.to/sergiidemianchuk/how-to-deploy-a-high-available-cluster-at-aws-opensearch-using-terraform-421i</link>
      <guid>https://dev.to/sergiidemianchuk/how-to-deploy-a-high-available-cluster-at-aws-opensearch-using-terraform-421i</guid>
      <description>&lt;p&gt;Hi, devops fans&lt;br&gt;
If you are interested at how to deploy HA OpenSearch cluster at AWS in details using terraformm then welcome to series of articles related to it:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt; &lt;a href="https://sergiiblog.com/terraform-opensearch-module-part-1/"&gt;TERRAFORM OPENSEARCH MODULE PART 1&lt;/a&gt;
&lt;/li&gt;
&lt;li&gt; &lt;a href="https://sergiiblog.com/terraform-opensearch-module-part-2/"&gt;TERRAFORM OPENSEARCH MODULE PART 2&lt;/a&gt;
&lt;/li&gt;
&lt;li&gt; &lt;a href="https://sergiiblog.com/terraform-opensearch-module-part-3/"&gt;TERRAFORM OPENSEARCH MODULE PART 3&lt;/a&gt;
&lt;/li&gt;
&lt;li&gt; &lt;a href="https://sergiiblog.com/apply-terraform-opensearch-module-domain-endpoint-and-opensearch-dashboard/"&gt;APPLY TERRAFORM OPENSEARCH MODULE, DOMAIN ENDPOINT AND 
OPENSEARCH DASHBOARD&lt;/a&gt;
&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;Have a pleasant reading :)&lt;/p&gt;

</description>
    </item>
    <item>
      <title>Terraform bastion module using autoscaling group</title>
      <dc:creator>Sergii</dc:creator>
      <pubDate>Fri, 17 Feb 2023 19:32:49 +0000</pubDate>
      <link>https://dev.to/sergiidemianchuk/terraform-bastion-module-using-autoscaling-group-492</link>
      <guid>https://dev.to/sergiidemianchuk/terraform-bastion-module-using-autoscaling-group-492</guid>
      <description>&lt;p&gt;Hi, devops fans&lt;/p&gt;

&lt;p&gt;As continuation of post related to "&lt;a href="https://sergiiblog.com/how-to-deploy-a-high-available-cluster-at-aws-opensearch/" rel="noopener noreferrer"&gt;How to deploy high available Elasticsearch cluster at AWS OpenSearch using terraform&lt;/a&gt;" a new articles appeared:&lt;/p&gt;

&lt;p&gt;&lt;a href="https://sergiiblog.com/terraform-bastion-module/" rel="noopener noreferrer"&gt;Terraform bastion module using autoscaling group&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;Have a pleasant reading&lt;/p&gt;

</description>
      <category>discuss</category>
    </item>
    <item>
      <title>Terraform AWS network module</title>
      <dc:creator>Sergii</dc:creator>
      <pubDate>Sat, 07 Jan 2023 10:03:46 +0000</pubDate>
      <link>https://dev.to/sergiidemianchuk/terraform-network-module-part-1-238p</link>
      <guid>https://dev.to/sergiidemianchuk/terraform-network-module-part-1-238p</guid>
      <description>&lt;p&gt;Hi, devops fans&lt;/p&gt;

&lt;p&gt;As continuation of post related to "&lt;a href="https://sergiiblog.com/how-to-deploy-a-high-available-cluster-at-aws-opensearch/" rel="noopener noreferrer"&gt;How to deploy high available Elasticsearch cluster at AWS OpenSearch using terraform&lt;/a&gt;" a new articles appeared:&lt;/p&gt;

&lt;p&gt;&lt;a href="https://sergiiblog.com/terraform-network-module-part-1/" rel="noopener noreferrer"&gt;Terraform network module – part 1&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;&lt;a href="https://sergiiblog.com/terraform-network-module-part-2/" rel="noopener noreferrer"&gt;Terraform network module – part 2&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;&lt;a href="https://sergiiblog.com/terraform-network-module-part-3/" rel="noopener noreferrer"&gt;Terraform network module – part 2&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;Have a pleasant reading&lt;/p&gt;

</description>
      <category>watercooler</category>
    </item>
    <item>
      <title>Initial terraform environment and pre-init terraform module</title>
      <dc:creator>Sergii</dc:creator>
      <pubDate>Mon, 26 Dec 2022 18:16:11 +0000</pubDate>
      <link>https://dev.to/sergiidemianchuk/initial-terraform-environment-and-pre-init-terraform-module-35fb</link>
      <guid>https://dev.to/sergiidemianchuk/initial-terraform-environment-and-pre-init-terraform-module-35fb</guid>
      <description>&lt;p&gt;Hi, devops fans&lt;/p&gt;

&lt;p&gt;As continuation of post related to "&lt;a href="https://sergiiblog.com/how-to-deploy-a-high-available-cluster-at-aws-opensearch/"&gt;How to deploy high available Elasticsearch cluster at AWS OpenSearch using terraform&lt;/a&gt;" a new articles appeared:&lt;/p&gt;

&lt;p&gt;&lt;a href="https://sergiiblog.com/what-is-opensearch-if-opensearch-and-elasticsearch-are-compatible/"&gt;What is OpenSearch? If OpenSearch and Elasticsearch are compatible?&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;&lt;a href="https://sergiiblog.com/initial-terraform-environment-and-pre-init-terraform-module/"&gt;Initial terraform environment and pre-init terraform module&lt;/a&gt;&lt;/p&gt;

</description>
    </item>
    <item>
      <title>How to deploy high available Elasticsearch cluster at AWS OpenSearch using terraform</title>
      <dc:creator>Sergii</dc:creator>
      <pubDate>Sun, 04 Dec 2022 13:35:38 +0000</pubDate>
      <link>https://dev.to/sergiidemianchuk/how-to-deploy-high-available-elasticsearch-cluster-at-aws-opensearch-using-terraform-2kl8</link>
      <guid>https://dev.to/sergiidemianchuk/how-to-deploy-high-available-elasticsearch-cluster-at-aws-opensearch-using-terraform-2kl8</guid>
      <description>&lt;p&gt;Hi, devops fans. Here I am opening the series of articles devoted to the theme of how to deploy high available Elasticsearch cluster using AWS service, which  is called OpenSearch, and terraform. The first article you already may read &lt;a href="https://sergiiblog.com/how-to-deploy-a-high-available-cluster-at-aws-opensearch/"&gt;here&lt;/a&gt;. New articles would be added gradually, so if you are interested at current topic, please visit my &lt;a href="https://sergiiblog.com/"&gt;blog&lt;/a&gt; regularly or &lt;a href="https://sergiiblog.com/sergii-blog-subscribe/"&gt;subscribe to my newsletter&lt;/a&gt;. But if you are not ready to wait – then I propose you to view all that material at my on-line course at udemy. &lt;strong&gt;&lt;a href="https://www.udemy.com/course/aws-devops-elasticsearch-at-aws-with-terraform-and-ansible/?referralCode=599D54E5628A86CAB801"&gt;Here is the link to the course&lt;/a&gt;&lt;/strong&gt;. &lt;/p&gt;

</description>
    </item>
    <item>
      <title>Elasticsearch basics: Elasticsearch head plugin, Postman and first DSL queries</title>
      <dc:creator>Sergii</dc:creator>
      <pubDate>Fri, 14 Oct 2022 16:30:49 +0000</pubDate>
      <link>https://dev.to/sergiidemianchuk/elasticsearch-basics-elasticsearch-head-plugin-postman-and-first-dsl-queries-fd5</link>
      <guid>https://dev.to/sergiidemianchuk/elasticsearch-basics-elasticsearch-head-plugin-postman-and-first-dsl-queries-fd5</guid>
      <description>&lt;p&gt;At my blog I am constantly getting questions which are related to elastisearch basics. That is why I decided to add several articles around that theme. Here is the 1st one article devoted to "&lt;a href="https://sergiiblog.com/elasticsearch-basics-elasticsearch-head-plugin-postman-and-first-dsl-queries/"&gt;Elasticsearch basics: Elasticsearch head plugin, Postman and first DSL queries&lt;/a&gt;". Have a pleasant reading.&lt;/p&gt;

</description>
    </item>
    <item>
      <title>Elasticsearch Basics: indexing and getting the 1st document</title>
      <dc:creator>Sergii</dc:creator>
      <pubDate>Fri, 30 Sep 2022 11:23:34 +0000</pubDate>
      <link>https://dev.to/sergiidemianchuk/elasticsearch-basics-4cdl</link>
      <guid>https://dev.to/sergiidemianchuk/elasticsearch-basics-4cdl</guid>
      <description>&lt;p&gt;At my blog I am constantly getting questions which are related to elastisearch basics. That is why I decided to add several articles around that theme. Here is the 1st one article devoted to "&lt;a href="https://sergiiblog.com/elasticsearch-basics-document-oriented-indexing-and-getting-first-document/"&gt;Elasticsearch Basics: document oriented nature, indexing and getting first document using curl and REST API, Elasticsearch data layer architecture&lt;/a&gt;". Have a pleasant reading.&lt;/p&gt;

</description>
    </item>
  </channel>
</rss>
