DEV Community: Alex Yaroslavsky

How to prevent a potential remote code execution via SnakeYAML deserialization

Alex Yaroslavsky — Thu, 05 Aug 2021 08:02:53 +0000

A popular java library for YAML parsing, SnakeYAML, has a well know vulnerability if used incorrectly to parse user generated YAMLs.

You can read about the vulnerability itself here:

SnakeYaml Deserilization exploited | by Swapneil Kumar Dash | Medium

Swapneil Kumar Dash ・ Sep 9, 2019 ・
swapneildash.Medium

The solutions for this problem that I have found on the net are either incorrect or unusable in real life. So I want to share here the solution that I have come up with.

It is quite simple:

  public static <T> T parseYamlSafe(String yaml, Constructor constructor) {
    Yaml yamlParser = new Yaml(new SafeConstructor());
    // the following line throws an exception
    // if constructors for non standard java types exist in yaml
    yamlParser.load(yaml);

    //if we got here, the YAML is safe to parse.
    yamlParser = new Yaml(constructor); 
    return yamlParser.load(yaml);
  }

Great article about tips for a better life

Alex Yaroslavsky — Fri, 01 Jan 2021 20:40:11 +0000

Link to article

Making Home and End keys work on Mac

Alex Yaroslavsky — Wed, 30 Dec 2020 08:27:34 +0000

Fix for most applications

Open a terminal window:

cd ~/Library
mkdir KeyBindings
cd KeyBindings
pico DefaultKeyBinding.dict

Copy the contents of the text below to the file, press Ctrl-X to exit the editor and select Y to save:

{
/* Remap Home / End keys */
/* Home Button*/
"\UF729" = "moveToBeginningOfLine:"; 
/* End Button */
"\UF72B" = "moveToEndOfLine:"; 
/* Shift + Home Button */
"$\UF729" = "moveToBeginningOfLineAndModifySelection:"; 
/* Shift + End Button */
"$\UF72B" = "moveToEndOfLineAndModifySelection:"; 
/* Ctrl + Home Button */
"^\UF729" = "moveToBeginningOfDocument:"; 
/* Ctrl + End Button */
"^\UF72B" = "moveToEndOfDocument:"; 
 /* Shift + Ctrl + Home Button */
"$^\UF729" = "moveToBeginningOfDocumentAndModifySelection:";
/* Shift + Ctrl + End Button*/
"$^\UF72B" = "moveToEndOfDocumentAndModifySelection:"; 
}

Fix for Terminal

Open terminal and go to preferences. Select the profile you use and go to the keyboard tab. Look for the the entries with actions \033b (home) and \033f (end) and edit them. When you edit them, select key Home or End respectively and modifier none.

Best explanation of Kubernetes services

Alex Yaroslavsky — Wed, 30 Dec 2020 08:16:59 +0000

Kubernetes Services simply visually explained | by Kim Wuestkamp | The Startup | Medium

Kim Wuestkamp ・ Jul 19, 2020 ・ 6 min read
Medium

Netstat without Netstat inside Containers

Alex Yaroslavsky — Thu, 23 Jul 2020 15:07:58 +0000

A very hardcore but simple way to view open connections and ports open for listening in any Linux container. Use the following command even inside any bare bones container without netstat and such tools:

grep -v "rem_address" /proc/net/tcp

The output will be something like this

   0: 00000000:1F40 00000000:0000 0A 00000000:00000000 00:00000000 00000000     0        0 6106109 1 ffff889f5ff35800 100 0 0 10 0
   1: 00000000:0C6D 00000000:0000 0A 00000000:00000000 00:00000000 00000000     0        0 6112091 1 ffff889ee7e96000 100 0 0 10 0
   2: 611F820A:0C6D E61E820A:A5E6 01 00000000:00000000 00:00000000 00000000     0        0 6122922 1 ffff889d2b712800 20 0 0 10 -1
   3: 611F820A:0C6D E61E820A:A5EE 01 00000000:00000000 00:00000000 00000000     0        0 6118270 1 ffff889e736e3000 20 4 21 10 -1
   4: 611F820A:1F40 F21F820A:DE3E 01 00000000:00000000 00:00000000 00000000     0        0 6119808 1 ffff889e78be7000 20 4 3 10 -1
   5: 611F820A:0C6D E61E820A:A5FC 01 00000000:00000000 00:00000000 00000000     0        0 6128773 1 ffff889e78bf2000 20 4 33 10 -1

The two left columns are source address and port and destination address and port in hex. The first two rows in this example correlate to listening ports (the destination is all zeros) and the rest are open connections.

To get this in a bit more human readable form you can run the following command that should also work inside most containers:

grep -v "rem_address" /proc/net/tcp | awk 'function hextonum(str, ret, n, i, k, c) {if (str ~ /^0[xX][0-9a-fA-F]+$/) {str = substr(str, 3);n = length(str);ret = 0;for (i = 1; i <= n; i++) {c = substr(str, i, 1);c = tolower(c);k = index("123456789abcdef", c);ret = ret * 16 + k}} else ret = "NOT-A-NUMBER";return ret} {y=hextonum("0x"substr($2,index($2,":")-2,2));x=hextonum("0x"substr($3,index($3,":")-2,2));for (i=5; i>0; i-=2) {x = x"."hextonum("0x"substr($3,i,2));y = y"."hextonum("0x"substr($2,i,2));} print y":"hextonum("0x"substr($2,index($2,":")+1,4))" "x":"hextonum("0x"substr($3,index($3,":")+1,4));}'

The output will be similar to this:

0.0.0.0:8000 0.0.0.0:0
0.0.0.0:3181 0.0.0.0:0
10.130.31.97:3181 10.130.30.230:42470
10.130.31.97:3181 10.130.30.230:42478
10.130.31.97:8000 10.130.31.242:56894

If you want to understand how this command works - leave a comment!

Multiple AWS accounts and CLI

Alex Yaroslavsky — Sat, 18 Jul 2020 11:54:17 +0000

After following this guide you will able to easily and seamlessly switch between multiple AWS accounts and roles (with or without Okta) using the CLI.

Prerequisites:

Check out the first article in the series for requirements and initial configuration.

AWS CLI with Okta

Alex Yaroslavsky ・ Jul 13 ・ 2 min read

#aws #cli #okta #productivity

How To:

The following instructions are meant to be used in Linux or WSL, tested with Ubuntu.

Configure non Okta accounts

aws configure --profile profile
aws configure --profile multi-role-profile

Configure Okta accounts

Create a file ~/.okta-aws with the following contents:

[okta-profile]
username = <username>
factor = OKTA
app-link = https://<your-company>.okta.com/<app-link>
base-url = <your-company>.okta.com
duration = 3600

[okta-multi-role-profile]
username = <username>
factor = OKTA
app-link = https://<your-company>.okta.com/<app-link>
base-url = <your-company>.okta.com
duration = 3600

Initialize the profiles:

okta-awscli --okta-profile okta-profile --profile okta-profile
okta-awscli --okta-profile okta-multi-role-profile --profile okta-multi-role-profile

Configure accounts with multiple roles

Some accounts might use role switching, add similar sections to ~/.aws/credentials per role (notice that source_profile points to a previously defined profile):

[multi-role-profile-role1]
role_arn = <role-arn>
source_profile = multi-role-profile

[okta-multi-role-profile-role1]
role_arn = <role-arn>
source_profile = okta-multi-role-profile

Associate EKS clusters with profiles

Run the following per EKS cluster that you want to have kubectl access to, <profile-name> is a name of the AWS profile defined above that has permissions for this EKS cluster:

aws --profile <profile-name> eks update-kubeconfig --name <eks-cluster-name>

Create scripts for fast account switching

The scripts switch to the relevant AWS account, point kubectl to the relevant cluster, and set a default kubectl namespace.
Create one script file per profile, and place it in your home directory.

File okta-multi-role-profile-role1:

export AWS_DEFAULT_PROFILE=okta-multi-role-profile-role1
kubectl config use-context <eks-cluster-arn>
kubectl config set-context --current --namespace=<namespace>
aws sts get-caller-identity
if [[ $PS1 != *"AWS_DEFAULT_PROFILE"* ]]; then
  PS1=\(\$AWS_DEFAULT_PROFILE\)$PS1
fi
echo "Switched to okta-multi-role-profile-role1"

Switch between accounts

To quickly switch between accounts just do the following:
source <profile-file>

For example:
source okta-multi-role-profile-role1

Spring Boot and Graceful Service Shutdown

Alex Yaroslavsky — Sat, 18 Jul 2020 10:24:20 +0000

Imagine your service running in Kubernetes, having the time of its life, and then suddenly you push a new version through the pipeline. Before starting the new version, Kubernetes needs to stop the old one. It first asks your service nicely to exit (SIGTERM), if your service doesn't respond to that it will be violently killed (SIGKILL) after several seconds. So if you want to close various resources correctly and exit gracefully, there are some steps you must take.

Below is an example of a generic skeleton for a Spring Boot Service that wants to handle shutdowns gracefully.

The main idea:

Start a separate thread with main application logic
Wait for application closing event (called upon SIGTERM)
Interrupt the main application logic thread and exit
PROFIT;

P.S. Don't forget to ~~eat~~ destroy your Beans, for Beans that implement the AutoClosable interface this is extremely simple:
@Bean(destroyMethod = "close")

AWS CLI with Okta

Alex Yaroslavsky — Mon, 13 Jul 2020 20:02:59 +0000

We will be using this solution: https://github.com/jmhale/okta-awscli

Prerequisites:

Python3
Windows: https://www.python.org/downloads/
AWS CLI V1
Windows: https://docs.aws.amazon.com/cli/latest/userguide/install-windows.html
Mac: https://docs.aws.amazon.com/cli/latest/userguide/install-macos.html
Linux: pip3 install awscli --upgrade --user

Install okta-awscli:

pip3 install okta-awscli

Initial setup:

Create aws profiles for dev and test.
Run the following:
aws configure set region us-east-1 --profile dev
aws configure set output text --profile dev
aws configure set region us-east-1 --profile test
aws configure set output text --profile test
Create the following file:
Linux: ~/.okta-aws
Windows: %USERPROFILE%.okta-aws
With the following contents:

[dev]
username =
factor = OKTA
app-link = <copy link from app icon in okta>
base-url = >your-company>.okta.com
duration = 3600

[test]
username =
factor = OKTA
app-link = <copy link from app icon in okta>
base-url = <your-company>.okta.com
duration = 3600

Login and run aws commands:

okta-awscli --okta-profile dev --profile dev

After logging in with okta-awscli, your login is valid for an hour and you can use aws commands (using the --profile )

Login to ECR:

aws --profile dev ecr get-login --registry-ids <your-ecr-id> --no-include-email

This will generate a token that you can use to login with docker to the ECR to pull images.
It will actually output the full command you need to run, so just copy it and run.
It will look like this:
docker login -u AWS -p https://<your-ecr-id>.dkr.ecr.us-east-1.amazonaws.com

Spring Boot and Multiple Authentication Profiles (None, Password & Okta)

Alex Yaroslavsky — Mon, 13 Jul 2020 16:02:58 +0000

Ahoy Spring user!

Recently, I added Okta login to my application to secure the admin section of the website. This is simple enough to accomplish but suddenly I felt the need to run my application locally without any authentication for testing purposes. As I couldn't find any good info on this on the web I decided to share the solution I came up with.

Full source code is here:

trexinc / spring-multi-web-security-config

Spring Boot and Multiple Authentication Profiles (None, Password & Okta)

A few quick words on setting up Okta authentication

Register for a free developer account at https://developer.okta.com/
Create a Spring Boot project with the following Spring Initialzer settings. Dependencies: Spring Web, Spring Security, Thymeleaf & OAuth2 Client. This solution works specifically with OAuth2 Client and not the native Okta library.
In Okta application configuration change the Login redirect URIs to: http://localhost:8080/login/oauth2/code/okta
In your application.properties configure the following:
And you need to define a Web Security Config to enable Okta authentication:

Back to the actual point

To enable several different security profiles I came up with the following solution. We define several Web Security Configurations each with its own @Profile("ProfileX) annotation, which enables this profile only when this profile is enabled in Spring.
The code looks likes this:

The relevant applications.properties sections look like this (of course only the one you want to use should be enabled, and it should be enabled only at runtime in your CI/CD pipeline):

What is the best way to run the code locally?

Once we have the code and know how to make it work one issue remains, what is the best way to run it locally with authentication disabled without modifying the application.properties files and risking pushing those modifications to productions.

Two options that I like the most:

Configure the relevant Spring profile in InteliJ, go to "Edit Configurations..." And set the profile as local
Configure a bootRun task in build.gradle that will set the profile to local (unless defined otherwise in the environment):

That's it. Hope you found this useful. Leave comments and likes :)

Integration testing Apache Pulsar clients with Docker Compose and TestContainers

Alex Yaroslavsky — Mon, 13 Jul 2020 05:59:40 +0000

Hello, quarantined user!

I would like to share with you my adventures with a complex Docker Compose testing scenario using TestContainers Java library that by pure coincidence (not really) also involves the coolest new kid on the block - Apache "The Kafka Killer" Pulsar.

Full source code is available at the end of the article.

Three main tools we are going to use today

Apache Pulsar

A distributed pub-sub messaging system with stream processing capabilities. It combines tenant isolation, seamless scaling and all the functionality of RabbitMQ and Kafka in one pretty damn badass package.

Docker Compose

A tool that allows you to keep multiple container configurations in a nice YAML file and to run multiple containers with one command.

TestContainers

A Java library for integration testing that can spin up any containers and let your tests interact with them in an easy and convenient way.

What are we trying to accomplish?

We want to set up an environment with the following components:

Apache Pulsar with three tenants: "internal", "customer1" & "customer2". All messages sent to the "customer" tenants will be routed by a Pulsar Function to a single topic in the "internal" tenant for easy centralized consumption.
Two Producers that will be sending messages to tenants "customer1" and "customer2" respectively - topic name: "/outbound/corona"
A single Consumer that will be receiving all messages from both Producers from a single topic "internal/inbound/corona"

It might seem that creating a single compose file with all the services above should give us what we need, but, it is not the case. First, we must start the Pulsar server, wait for it to load, configure all the tenants and the Functions. Then, we should start the consumer so it will be ready to read the messages and then finally start the producers. After all the components are up and running we want to verify that the producers are sending messages and the consumer is receiving them.

Can we do it manually first?

Before trying to automate the task, let's figure out how to do it manually using only Docker Compose.

We already understand that we will need several compose files and we know that all the components must be able to communicate with each other, which means that they have to be on the same network. Docker Compose allows just that. We can define a network in one compose file and then use it in another compose file by marking it there as external.

Here you can see a Pulsar compose file that defines a network called "local-pulsar-net":

And here is a different compose file that uses the same network for its service:

We see above that the "pulsar-net" network that is defined in this compose file is linked to an external network called "local-pulsar-net", which happens to be the network we defined in the previous compose file.

Running those compose files with docker-compose up will place the containers in the same network and they will be able to communicate with each other by their respective service names.

Let's automate this coronapulsar!

TestContainers library is quite powerful and easy to use, here is an example code snippet showing how to start a Pulsar server from a compose file:

Until this point all is great, we can even create a PulsarAdmin client and configure our Pulsar the way that we need. Unfortunately, starting the second compose file that references the network defined in the Pulsar compose file just fails with an exception:

What happens is that TestContainers adds a random prefix to all components it loads to Docker. This is of course necessary to prevent name collisions and such when loading many different compose files. So, for example, our "local-pulsar-net" is created as "pulsarrpekn3_local-pulsar-net" in the run above. And the name will be different in the next one and so on…

Let's solve this COVID-bug!

It seems that in order to solve our issue we need:

To know the random name of the "local-pulsar-net" network as generated by TestContainers.
To somehow apply this name to the existing additional compose files before loading them.

Fortunately, there is way to do both. Hooray!

Getting the name of the network:

To apply the network name to the remaining compose files we can use the following trick, define the network like this in the YAML files:

note the last line - name: $PULSAR_NETWORK

This allows to pass the name of the network as an environment variable when starting the compose file, like this:

The final chapter

You can find the full source code that starts and configures a standalone Pulsar, then starts a Consumer, two Producers, verifies that they are all working correctly and shuts down the environment on my GitHub page:

trexinc / pulsar-integration-testing

Integration testing Apache Pulsar clients with Docker Compose and TestContainers

Thank you for reading! Like the page and leave some comments, hope this article was useful to you.

Stay healthy & safe!

DEV Community: Alex Yaroslavsky

How to prevent a potential remote code execution via SnakeYAML deserialization

SnakeYaml Deserilization exploited | by Swapneil Kumar Dash | Medium

Swapneil Kumar Dash ・ Sep 9, 2019 ・ swapneildash.Medium

Great article about tips for a better life

Making Home and End keys work on Mac

Fix for most applications

Fix for Terminal

Best explanation of Kubernetes services

Kubernetes Services simply visually explained | by Kim Wuestkamp | The Startup | Medium

Kim Wuestkamp ・ Jul 19, 2020 ・ 6 min read Medium

Netstat without Netstat inside Containers

Multiple AWS accounts and CLI

Prerequisites:

AWS CLI with Okta

Alex Yaroslavsky ・ Jul 13 ・ 2 min read

How To:

Configure non Okta accounts

Configure Okta accounts

Configure accounts with multiple roles

Associate EKS clusters with profiles

Create scripts for fast account switching

Switch between accounts

Spring Boot and Graceful Service Shutdown

AWS CLI with Okta

Prerequisites:

Install okta-awscli:

Initial setup:

Login and run aws commands:

Login to ECR:

Spring Boot and Multiple Authentication Profiles (None, Password & Okta)

trexinc / spring-multi-web-security-config

Spring Boot and Multiple Authentication Profiles (None, Password & Okta)

A few quick words on setting up Okta authentication

Back to the actual point

What is the best way to run the code locally?

Integration testing Apache Pulsar clients with Docker Compose and TestContainers

Three main tools we are going to use today

Apache Pulsar

Docker Compose

TestContainers

What are we trying to accomplish?

Can we do it manually first?

Let's automate this coronapulsar!

Let's solve this COVID-bug!

The final chapter

trexinc / pulsar-integration-testing

Integration testing Apache Pulsar clients with Docker Compose and TestContainers

Swapneil Kumar Dash ・ Sep 9, 2019 ・
swapneildash.Medium

Kim Wuestkamp ・ Jul 19, 2020 ・ 6 min read
Medium