DEV Community

Cover image for How To Monitor Http traffic Of Third Party Service
Yw Zhang
Yw Zhang

Posted on

How To Monitor Http traffic Of Third Party Service

Why

The company purchased a third-party service and deployed it on the company's machine in the form of a jar package.This service is on a critical data link, and the concurrency is very high. The service http request appears 500, which is what we cannot receive.

Even without the source code, we also need to monitor HTTP requests to provide detailed information to the provider.

Ideas to Solve the Problem

My first thought was to use java agent to monitor.Java agents are a special type of class which, by using the Java Instrumentation API, can intercept applications running on the JVM, modifying their bytecode.

Another solution is to capture network traffic. My core requirement is to monitor requests with a status code of 5xx, so capturing traffic is more suitable.

Capture network traffic

Tcpdump can capture traffic but needs parsing to be more suitable for reading.

httpry is a tool designed for displaying and logging HTTP traffic.

Nice!This is the tool I was looking for.

Install httpry on Centos

sudo yum install epel-release

sudo yum install httpry
Enter fullscreen mode Exit fullscreen mode

Filter Response with a Status Code other than 200

sudo httpry -m post -f timestamp,source-ip,dest-ip,direction,status-code -o test.log 'tcp dst port 80 and tcp[((tcp[12:1] & 0xf0) >> 2)+8:4] != 0x20323030'
Enter fullscreen mode Exit fullscreen mode

The output in test.log like this:

2022-09-01 19:22:34     192.168.0.1  192.168.0.2 <       500
2022-09-01 19:22:34     192.168.0.1  192.168.0.2 <       500
2022-09-01 19:22:34     192.168.0.1  192.168.0.2 <       500
2022-09-01 19:22:34     192.168.0.1  192.168.0.2 <       403
2022-09-01 19:22:34     192.168.0.1  192.168.0.2 <       500
2022-09-01 19:22:34     192.168.0.1  192.168.0.2 <       403
2022-09-01 19:22:34     192.168.0.1  192.168.0.2 <       500
2022-09-01 19:22:34     192.168.0.1  192.168.0.2 <       500
Enter fullscreen mode Exit fullscreen mode

Manager Large Data Volume Output File

All the information is typed into one file, this file will be very large.

You need another tool:logrotate.

I want to organize the files by date and I create the file logrotate.conf

/xxxx/xxxxx/http.log {
    notifempty
        copytruncate
        dateext
        dateformat .%Y%m%d
        olddir /xxx/backup
}
Enter fullscreen mode Exit fullscreen mode

The following command can debug the process

logrotate -d xxx/lograte.conf
Enter fullscreen mode Exit fullscreen mode

Then add timed tasks through ctontab,crontab will execute the rotate according to the config and We will get a file named http.log.20220815.

More Resources

httpry
logrotate

Top comments (0)