With its ability to provide a unified view of your application’s health, backed up by data in the form of logs, metrics and events, CloudWatch proves AWS’s leading position in cloud computing. That being said, CloudWatch has its pain points. Here are ten commands to help you through any rough spots. They run on the AWS CLI, which can be installed on Mac, Windows and Linux.
Logs
CloudWatch’s logging system has two fundamental units. Log streams are sequences of logging events that come from a single source. Log groups are convenient boxes to put bundles of log streams in.
create-log-group
This allows users to create log groups to put log streams in. When a log group is created, its default behaviour is that log events never expire.
Log group names can be between 1 and 512 characters in length and must consist entirely of valid characters. These are lowercase and capital letters a-z, the numbers 0-9, '_' (underscore), '-' (hyphen), '/' (forward slash), '.' (period), and '#' (number sign).
Additionally they need to be unique within a given AWS account’s geographic region. To explain what this entails, we need to understand the structure of an AWS account. When a user creates an account, its AWS resources are parcelled up into geographic areas called regions and each account can have multiple regions enabled.
When you create a log group for the region ‘Europe (Milan)’, for example, your account must have no other log groups of the same name in ‘Europe (Milan)’.
create-log-stream
This enables users to create a log stream for a particular log group. Users can decide what to call the log stream with the --log-stream-name option and they can use the --log-group-name option to attach the stream to a log group of their choosing. Valid log stream names in AWS have to conform to three criteria.
First, they need to have a length between 1 and 512 characters. Second, they must be unique within the log group that they are associated with. Third, they cannot contain colons or asterisks.
describe-log-groups
This command allows the user to see at a glance the log groups in their system. When the command returns results, they are ASCII-sorted by log group name.
This means that log group names are displayed in alphabetical order with the added property that names beginning with capital letters universally appear before names beginning with lower case characters. Names beginning with numbers are displayed before names beginning with letters.
The command comes with a suite of options allowing users to view as many or as few log groups as they wish, limit the results by prefix and specify page size. If pagination is not to your liking, you can disable it with the --no-paginate option.
describe-log-streams
This command allows the user to see at a glance the log streams in a given log group. Results are paginated by default to make extensive lists easier to browse, plus there are several command options that allow users to order log stream events by name or by the time they occurred, as well as viewing as many or as few events as appropriate.
filter-log-events
This command enables users to view the log events associated with a given log group. If no log group is specified, the command spits out as many log events as it can find, up to a maximum of 1MB worth.
An arsenal of versatile options enable users to filter log events through a range of different criteria. The --log-group-name option is useful to users who want to view log events from a specific log group.
If users want to filter events by log stream, they have a choice between --log-stream-names and --log-stream-name-prefix. The --log-stream-names option lets users view events from one or more particular log streams while the --log-stream-name-prefix option displays events from the log streams starting with a specific character string, e.g all the log streams that started with “abc123”.
The start-time and end-time options display events that occurred within a given time range.
get-log-events
This command allows users to quickly access the content of log streams and view a list of log events. The --log-group-name and --log-stream-name options enable users to specify the exact log stream or log group that their log is in.
The command is highly versatile, allowing users to specify how many events they want to view, the order in which they want to view them and the time range they are interested in looking at. The --log-group-name option allows log events to be filtered by log group.
get-log-record
Enables users to view the fields and values of a single log event. The --log-record-pointer option allows a user to pinpoint the exact log event they wish to view.
Seeing the Bigger Picture
Essential to DevOps is the ability to monitor the overall health of an application. DevOps engineers need to be able to catch any fires burning in their system before they reach their flashpoint. CloudWatch has two useful tools for this purpose, dashboards and metric statistics.
get-metric-statistics
This enables users to easily see the statistics for a specified metric. The AWS documentation defines statistics as “metric data aggregations over specified periods of time”.
A number of useful options allow users to display the exact metric statistics they want to see. The first two of these are --metric-name and --namespace. metric-name takes the name of the metric a user wants stats for but namespace is a little more complicated.
An AWS namespace is like a hermetically sealed box that metrics live in. Each metric knows only its home namespace and nothing about metrics in other namespaces. Because there is no default namespace, a user who wants information on a given metric must always specify the metric’s namespace.
Another interesting option is --dimensions. AWS metrics are complex data structures that can contain up to ten dimensions. The --dimensions option lets users list all the dimensions of a specified metric.
There are three options that control time ordering of metric statistics. The --start-time and --end-time options, as with other commands, define the beginning and end of a given time range that the user is interested in returning data for. The third option is called --period and controls the sampling frequency of returned data.
AWS provides six statistics for each metric.
Minimum shows the lowest value of a metric data sample within a specific time range while maximum shows the highest value.
Sum refers to the values of a particular metric added together and can be useful for calculating metric volume. SampleCount signifies the number of data points used in statistical analysis.
Average is a statistic defined by the expression Sum/SampleCount for a specified time range. This statistic can be extremely useful in combination with Maximum and Minimum and can greatly aid tasks such as resource allocation.
pNN.NN is the value of a given percentile. These statistics can be viewed in raw form using the
--statisticsoption, or in percentile form with--extended-statistics.
There we are!
Cloudwatch isn't perfect and is lacking in many of the features that other monitoring, logging and observability tools have, but with these commands, you'll be able to easily traverse the often complex world of AWS monitoring.
Top comments (0)