DEV Community: Louis Leung

New Instant Observability integrations for monitoring API, network, application, and streaming performance

Louis Leung — Sun, 31 Mar 2024 22:01:59 +0000

To read this full New Relic blog, click here.

Engineers rely on many tools to monitor the health of their systems, making it difficult to gain end-to-end observability without having to toggle between multiple solutions. To help combat this issue of tool sprawl and data silos, we partner with leading technology companies to build integrations to make it easy to visualize, monitor, and act on all your data in a single observability platform.

You can find all of our partner integrations in New Relic Instant Observability (I/O), an open ecosystem of more than 450 free, prebuilt quickstarts that help you start monitoring your system in minutes. These quickstarts bundle necessary building blocks for instrumenting, monitoring, and acting on signals from your telemetry data, wherever it comes from.

New partner integrations

The latest cohort of partners contributing to New Relic Instant Observability span across customer experience monitoring, web performance, and API management.

This month, we’re announcing new integrations and quickstarts with the following partners:

Datazoom
Glassbox
Cloudflare
Netlify
Postman

Read on to learn what you can do with these new quickstarts.

Cloudflare Network Logs

Cloudflare is an industry-leading global network designed to make everything you connect to the Internet secure, private, fast, and reliable. Use Cloudflare to protect and accelerate external, public-facing web properties, secure your internal operations on one global network, and build new applications on Cloudflare's serverless platform. Internet properties powered by Cloudflare have all web traffic routed through Cloudflare's intelligent global network.

The new Cloudflare Logpush integration sends Cloudflare data directly into New Relic, resulting in faster log delivery and eliminating cloud storage middleware costs. With the Cloudflare Network Logs quickstart, you can monitor and analyze web traffic metrics from Cloudflare Logpush on a New Relic dashboard. You get an overview of important logs and metrics from your websites and applications.

Learn more about how to send log insights faster and cheaper through the Cloudflare integration.

Netlify Logs

Platform developers love Netlify for building high-performance, dynamic web sites, e-commerce stores, and applications. As a pioneer of the Jamstack movement, Netlify brings together modern web frameworks, serverless functions, and edge computing into one platform to deliver unmatched experiences for your application users.

With the Netlify Logs quickstart, you can explore and visualize data from traffic and function logs provided by Netlify log drains. It allows you to parse, view, and filter traffic logs and function logs generated by your Netlify workloads for in-depth analysis, performance monitoring, troubleshooting, and alerting within New Relic.

Learn more about exporting traffic and function logs from Netlify to New Relic by reading the Better monitoring for your Netlify sites blog post, or watch this tutorial:

Postman

Postman helps more than 17 million developers collaborate and create better APIs. The Postman quickstart provides Postman monitoring data to New Relic, so that DevOps, APIOps, and ITOps engineers can get instant observability into critical metrics such as latency, request counts, and error rates to collaborate better. The integration also ingests data from New Relic into Postman to bring observability data to API producers, testers, and consumers.

Learn how to send real-time API metrics from Postman to New Relic in the API monitoring with Postman integration blog post, or watch this tutorial:

Datazoom

Datazoom is an enterprise video data platform. The software solution captures, standardizes, enriches, and routes data from various points across an end-to-end video workflow. Through a variety of real-time data collection and routing services, Datazoom’s platform provides flexibility and transparency in data collection so that teams can make operations, engineering, product and business decisions with confidence. Companies can drive revenue with streaming video by using Datazoom to democratize insight, increase efficiencies, and deliver captivating end-user experiences.

The Datazoom quickstart provides a fast and easy launching point into video metrics built on top of New Relic dashboards, so you can gain insights about your viewers, content, ads, and your platform's performance. You can customize the dashboard to use only metrics that fit your goals, edit metric calculations, and tailor them to your needs.

Learn more about the Datazoom integration at Get real-time visibility into your streaming video architecture and performance.

Glassbox

Glassbox empowers organizations to create frictionless digital journeys for their users. The Glassbox digital experience analytics and session replay platform work in real time across mobile apps and websites to increase user loyalty and accelerate growth. Through AI-driven visualization and analytics tools, Glassbox helps teams to prioritize customer experience and digital product enhancements from a single collaborative system.

The Glassbox quickstart helps you understand the impact of application performance on your users’ digital experience. Gain deeper contextual insights by combining the data capture and session replay capabilities of Glassbox with New Relic. Get a view of behavioral insights and KPIs inside a pre-built New Relic dashboard, which links directly to a session replay in Glassbox, so you can find the root cause of the digital issue and fix performance issues faster.

To read this full New Relic blog, click here.

Get your insights in minutes

Louis Leung — Sun, 31 Mar 2024 21:40:18 +0000

To read this full New Relic blog, click here.

At New Relic, we are committed to making observability a daily best practice for every engineer by investing heavily in the global open source community. In the last 12 months alone, we open sourced more than 10 years of R&D around agents, we standardized on OpenTelemetry, and we contributed Pixie to the Cloud Native Computing Foundation. New Relic I/O is a continuation of this strategy to dramatically reduce the barrier for engineers to embrace observability. Anyone can contribute data sources and share alert and dashboard configurations. Ultimately, we created New Relic I/O to make it easier for engineers to leverage community expertise to get more value out of their data, instantly.

The democratization of observability

How do you know that you are properly monitoring your LAMP stack, getting the right visibility into the performance of Gatsby sites, or debugging Kafka clusters with proven best practices? Today’s distributed environments are complex, and most engineers aren’t observability experts. Unless you’ve done it before, it’s difficult and time consuming to figure out how to instrument your distributed tech stack, capture the right data points with the right detail, and build dashboards from scratch to monitor your telemetry data.

According to New Relic’s 2021 Observability Forecast, while most IT decision makers are familiar with observability, there’s a huge gap in the practice: only 26% of respondents have a mature observability practice, and the most commonly cited barriers to observability success are lack of resources (38%) and skill gaps (29%).

What if there was an easier way to share these observability superpowers with all software engineers? And to get started quickly?

A single ecosystem for integrations, applications, and quickstarts

Enter New Relic I/O, an ever-expanding, open knowledge base of configurations and resources that taps into the collective experience of the world’s observability experts to help you unlock the value of your data faster. You can choose from hundreds of quickstarts (including LAMP, Gatsby, and Kafka) that bundle the necessary building blocks for instrumenting, monitoring, and acting on signals from your technology stack—and install them in a click. We’ve also built quickstarts for hundreds of end-to-end integrations for popular cloud services, tools, and open standards that include guided installation experiences as well as relevant dashboards and alerts. No matter which technologies you rely on, you can get observability coverage in minutes instead of days—or weeks.

Within New Relic Instant Observability, current New Relic One users can extend the platform with publicly available apps or any custom apps your team builds.

If you haven't used New Relic before, you can browse the catalog of available quickstarts in our public New Relic I/O catalog and get started with a free account in minutes.

How to install your first quickstart

Getting started with observability has never been easier. Before you begin, if you haven't used New Relic before, sign up for a free New Relic account (or log in to your existing account).

In New Relic One, to see the catalog, click Instant Observability in the top navigation. You can search, filter by contents, or select a category, such as Kubernetes or APM.

Then select a quickstart to learn more about it:

After you select a quickstart, you can see what’s included in the quickstart details. See a description of the pack, screenshots, and information on the included resources. Click Install this quickstart to get started.

Follow the instructions and install the necessary instrumentation to get the data used in the quickstart:

After you complete the installation process, click See your data:

Enjoy the dashboards, alerts, and insights:

Voila! You’re on the path to becoming an observability expert. Yes, it’s really as simple as that.

Partners in instant observability

We are proud to launch New Relic Instant Observability with pre-built quickstarts from six leading enterprise software partners. We have partnered closely with them to create quickstarts that help you extend your New Relic One experience:

Kentik is the network observability company. The Kentik quickstarts help network and development teams quickly identify and troubleshoot application performance issues correlated with network traffic performance and health data.
Fastly is an edge cloud platform that enables its customers to create great digital experiences quickly, securely, and reliably. With the Fastly CDN quickstart, you can monitor key metrics from Fastly’s content delivery network to improve service reliability and ensure great online experiences for end users.
Lacework is a data-driven security platform for the cloud that can collect, analyze, and accurately correlate data across an organization’s Amazon Web Services (AWS), Microsoft Azure, Google Cloud Platform, and Kubernetes environments, and narrow it down to the handful of security events that matter. The Lacework quickstart bridges the gap between observability and security teams, and integrates with New Relic’s database to surface security events and alerts directly in New Relic One.
Cribl is the observability pipeline company that lets customers parse and route any type of data. The Cribl quickstart allows you to get immediate visibility into your entire environment right from New Relic One without the need to create your own dashboards and alerts—simplifying your workflows and reducing time to value.
Trend Micro is a global cyber security leader. The Trend Micro Cloud One quickstart ingests cloud security posture management (CSPM) data from Conformity into New Relic One to contextualize and correlate it with workload telemetry data, delivering AI-powered visualizations and quick insights. This allows security and cloud teams to immediately take action in improving their security and compliance postures.
Gigamon is a hybrid-cloud visibility and analytics platform that provides access to—and extracts intelligence from—all network traffic. The Gigamon quickstart delivers advanced security capabilities that offer network detection and response to advanced threats, including shadow IT activities, crypto-mining and torrent activities, SSL cipher versions, and expiration dates across both managed and unmanaged hosts, such as IoT/OT and containers.

To read this full New Relic blog, click here.

Partners in Instant Observability: Integrations for machine learning, Kubernetes, CI/CD

Louis Leung — Sun, 31 Mar 2024 21:14:09 +0000

To read this full New Relic blog, click here.

We recently announced the launch of New Relic Instant Observability (New Relic I/O), an open ecosystem of free, pre-built quickstarts that help engineers start monitoring their stacks in minutes without manual setup. New Relic I/O includes more than 425 quickstarts that bundle necessary building blocks for instrumenting, monitoring, and acting on signals from your telemetry data from the technologies you rely on, such as Kubernetes, CodeStream, and PHP.

Alongside the quickstarts for popular cloud services, tools, and open standards, leading enterprise software partners also contributed their own integrations and quickstarts to New Relic I/O, making it easy to visualize and monitor all your data in one platform.

New partner quickstarts

The latest cohort of partners contributing to making observability a daily habit for engineers span across CI/CD, Kubernetes testing, and leading machine learning operations (MLOps) platforms. These partners contributed quickstarts in the past month:

Delphix
Speedscale
Amazon SageMaker
Algorithmia
Aporia
Comet
DAGsHub
Monalabs
Superwise
Truera

Learn more about each of the companies and quickstarts in the next sections.

Instant observability for CI/CD

Delphix is an industry leading data company for DevOps. Their platform provides essential APIs for data provisioning, refresh, rewind, integration, and version control. The Delphix quickstart allows you to get visibility into all your Delphix Virtualization platforms on premises or in the cloud, so that you can rapidly identify and reproduce issues, perform root causes analysis, and dramatically reduce MTTR and downtime.

Instant observability for Kubernetes testing

Speedscale helps Kubernetes engineering teams gain confidence in how new code will perform in real-world scenarios. It provides traffic replay capabilities that help developers discover API issues earlier in their release cycle. Collect and replay API traffic, simulate load or chaos, and measure latency, throughput, saturation, and errors before the code is released. The Speedscale quickstart visualizes traffic replay data into New Relic One dashboards so you can compare this with your telemetry data in New Relic One.

Instant observability for AI/ML model monitoring

New Relic also recently extended our observability experience to provide a new offering for artificial intelligence (AI) and machine learning (ML) teams to break down visibility silos with model performance monitoring. With quickstarts for each of the available MLOps integrations, it’s easy to get complete visibility into ML-powered applications.

Amazon SageMaker enables developers to create, train, and deploy machine-learning (ML) models and to deploy ML models on embedded systems and edge devices. SageMaker supports leading machine learning frameworks, toolkits, and programming languages. With the Amazon SageMaker quickstart you can get a view of the performance metrics, visualize sophisticated ML models, and create a comprehensive monitoring dashboard in New Relic One for your AI/ML applications pipelines.

Algorithmia manages all stages of the production ML lifecycle and accelerates delivery of your ML models into production. It guides a process of continuous optimization across all stages of the ML lifecycle within existing operational processes. With the Algorithmia quickstart, you can instrument, analyze, troubleshoot, and optimize your machine-learning performance across your entire system.

Aporia provides a fast, easy, and secure way for data science teams to monitor their machine learning models in production. Teams can build their own customizable monitors in minutes to receive live alerts for early detection of issues like data drift, unexpected bias, data integrity issues, and performance degradation. With the Aporia quickstart, data scientists and ML engineers receive a combination of cross-platform and integrated smart alerts. These provide a broader picture of the ML infrastructure and an easy, quick way to resolve issues.

Comet is an ML development platform built to meet the intense demands of enterprise teams deploying ML at scale. With Comet’s quickstart, you can manage and optimize your ML models in production from your New Relic One dashboard. Using Comet and New Relic One together, you can accelerate ML development and realize business value faster.

DAGsHub is a platform for data scientists and machine learning engineers to version and sync their data, models, experiments, and code. Your team can easily share, review and reuse your work, providing a GitHub-like experience for machine learning. DagsHub is built on popular open-source tools and formats, making it easy to integrate with the tools you already use like New Relic One. With the DAGsHub quickstart, you can analyze and monitor machine learning training metrics in real-time.

Mona provides complete visibility into your data and models in the real world with the most flexible and scalable AI monitoring platform. Mona’s integration with New Relic One provides next-level monitoring capabilities under a unified production AI environment, from ML models to infrastructure monitoring. With the Mona quickstart, you instantly get alerted on latency issues. It automatically detects anomalies to ensure the best performance within production systems.

Superwise.ai is the company that monitors and ensures the health of AI models in production. The model management platform leverages machine learning at high scale. Their AI assurance solution acts as the one source of truth for all stakeholders. It empowers data science and operational teams with the right insights to scale their use of AI by gaining confidence in their models’ operations and becoming more independent and agile. With the Superwise quickstart, you gain observability and get preconfigured ML monitoring in New Relic One.

TruEra provides best in class AI quality diagnostics and monitoring. With the TruEra quickstart, you go beyond standard accuracy, input and output drift monitoring to identify consequential data drift and other AI quality diagnostics such as fairness, conceptual soundness, and segment behavior. When combined with the comprehensive observability capabilities of New Relic One, MLOps teams can correlate signals throughout the ML lifecycle.

Bring your own ML data into New Relic One

Have another ML stack that you want to integrate with New Relic One? Use the ML Model Monitoring quickstart to bring your own ML model data from any source and integrate ML data with the rest of the application components, including infrastructure.

To read this full New Relic blog, click here.

Automate root cause analysis in logs with New Relic and Zebrium

Louis Leung — Tue, 12 Mar 2024 04:28:29 +0000

To read this full New Relic blog, click here.

Determining the root cause of an incident has become more complex as engineers increasingly rely on distributed microservices to power their applications. Zebrium Root Cause as a Service, together with New Relic, helps automate the process of finding the root cause of issues in your logs. When a software or infrastructure problem occurs, Zebrium quickly and accurately finds the root cause indicators so your teams don’t need to manually search through all their logs. With the Zebrium integration for New Relic, you can see root cause details on any New Relic dashboard page, so you can leverage your telemetry data to help identify and troubleshoot issues faster.

To make it easier to get started quickly, the Zebrium quickstart in New Relic Instant Observability (I/O) provides a pre-built dashboard that includes incident detections, root cause details, and deep links to the Zebrium user interface. Here’s an example of the dashboard in New Relic.

AI-Powered Root Cause Analysis with Zebrium Integration | New Relic

Any engineer can get started for free with New Relic and use the feature as part of the more than 470 integrations available with New Relic’s observability platform. When you sign up, you get 100GB per month of free data ingest, one full platform user, and unlimited basic users, queries, dashboards, and alerts.

With the Zebrium integration in place, you can find the root causes of problems more quickly, reducing the burden on your SREs and DevOps teams when solving complex incidents. You get the following key benefits:

Never dig through logs again. When a problem occurs, automatically see log lines related to the issue in your New Relic dashboards. This can reduce downtime and speed up resolution from hours to minutes.
No manual training or rules needed. Getting started with initial setup only takes about 15 minutes, and Zebrium’s root cause as a service achieves accuracy in finding root cause indicators from log data within just 24 hours. No need to set up machine learning (ML) training manually!
Accurately find root causes with confidence. A recent third-party customer case study validated that Zebrium can automatically find the root cause in over 95% of incidents.

Identify root causes at a glance with the Zebrium integration

Let’s take a look at how the integration can help you identify root causes without needing to hunt through logs manually. The dashboard in the next image shows metrics for an online shopping app on a Kubernetes cluster. The two time-series metrics charts at the top of the dashboard show that an outage occurred.

The Zebrium Root Cause Finder chart on the bottom left shows a vertical bar at the time the problem occurred, indicating Zebrium’s machine learning model has detected a potential issue. The vertical bar is correlated with the drop in metrics. Meanwhile, the Zebrium Root Cause Reports chart on the bottom right details the Zebrium event. You can mouse over the Zebrium event to see a natural language processing (NLP) summary and word cloud that can help you determine the root cause of the issue. Selecting the Deep Link Url opens the full root cause report in the Zebrium user interface.

As shown in the previous image, you can dive into the Zebrium UI for the root cause report. Approximately one million log lines were generated during the incident time window. The report highlights 46 lines from 7 different services, and no manual model training or rules were required to find these log lines. A quick review of the report reveals:

An English language NLP summary that gives a good sense of the problem: “The Chaos Monkey was trying to create an order.”
A word cloud showing “pod-network-corruption” and some other relevant words.
Log lines that provide descriptions of the root cause.

Zebrium uses statistical machine learning to analyze logs in real time. The machine learning model is based on the process a skilled engineer uses to search through logs when troubleshooting. In simple terms, this typically involves searching for errors around the time an issue occurs and then searching all logs around that time period to find any rare or unexpected events that explain the problem.

How to get started with automated root cause identification in minutes

To start viewing root causes from Zebrium events automatically in New Relic, watch the following video or follow the steps listed after the video. The whole setup process should take about 15 minutes.

Set up your accounts. You need to have New Relic and Zebrium accounts to use the integration. You can get started for free.
Install a log collector to stream your logs to Zebrium. Read more about the supported collectors and how to install them.
Retrieve your New Relic API key. Go to API keys in New Relic. Select Create a key. Choose your account and note the Account Id. Select Ingest - License as the Key type. Enter a name and notes for the API key, then select Create a key. Select the three dots next to your new key and select Copy key to save it for later.
Enable the integration in Zebrium. Go to the User menu in Zebrium, select Integrations & Collectors, and select New Relic under the Observability Dashboards section. Select the General tab and enter an integration name. Select the Deployment for the integration, select Service Group(s), and then select the Send Detections tab. Select Enabled, and enter the Account Id and API key from the previous step. Then select Save.
Visualize your data with the quickstart. After you’ve enabled the integration to send detections from Zebrium to New Relic, install the quickstart to get a curated dashboard. Go to the Zebrium Root Cause as a Service quickstart in New Relic I/O, select the Install Now button, and follow the guided installation process.

After you are finished, you automatically get a curated sample dashboard in New Relic where you can view detections and their root causes. You can also copy the sample charts into any other New Relic dashboard for a customized view.

The next time you have to deal with a P1 incident, you can automatically see what happened directly in your New Relic dashboards. Zebrium Root Cause as a Service (RCaaS) can be set up in minutes and achieves correlation accuracy within 24 hours without any rules or manual training. Find the root cause faster and never dig through logs again.

To read this full New Relic blog, click here.

Create and monitor WayScript self-service dev environments in minutes

Louis Leung — Thu, 29 Feb 2024 23:20:45 +0000

Software development demand—and complexity— is increasing every day, creating a growing challenge for SRE and DevOps teams to keep up and simultaneously support ad hoc internal development needs. Interrupting the SRE workflow to provision and manage environments is costly. Self-service infrastructure for developers can solve this problem, but can be a massive headache to implement and manage.

To tackle this developer self-serve challenge, New Relic and WayScript built an integration that lets you quickly create a developer environment and automatically monitor it with a suite of observability tools. With WayScript and New Relic, you can:

Instantly spin up containerized developer environments in a single click.
Host full-fledged apps, set up internal microservices, and more, with built-in tools like a terminal and IDE.
Monitor and troubleshoot everything you build with an easy connection to the New Relic observability platform.

All developers can get started for free with New Relic and connect with WayScript as part of the more than 480 integrations included with New Relic’s observability platform. Free accounts with New Relic receive 100 GB per month of free data ingest, one full platform user, and unlimited basic users, queries, dashboards, and alerts. With preconfigured cloud-native environments from WayScript and rich observability capabilities from New Relic, you can focus on writing code instead of managing infrastructure.

Automate development environment monitoring with New Relic and WayScript

WayScript is an internal developer platform (IDP) that makes it easy for software developers, SREs, and engineering leaders to take control of their team’s tools and developer environments. You can create self-service, cloud-based developer environments that scale, so SREs don't need to disrupt their workflows to provision one-off environments. WayScript is SOC-2 Type II certified. You can use it on the browser, the desktop app, or self-host WayScript on your own EC2 instance.

Developers can quickly build cron tasks, services, APIs, servers, data pipelines, and any other internal tool, without needing to learn new platforms and technologies. The cloud-based Docker environments come with built-in SSO, Kubernetes, internal data access, and now New Relic monitoring.

Monitor Self-Serve Developer Environments with WayScript | New Relic

With this new integration, you can automatically initialize New Relic to configure application performance monitoring (APM) agents, log forwarding, and more during the initial workspace setup in WayScript. You’ll get automatic visibility into the health of all pre-production, staging, and production environments.

How to create and monitor your environment in minutes

Now let’s try it out. There are just four main steps for connecting WayScript and New Relic:

Open a WayScript workspace for your team.
Create your first Lair for a containerized development environment.
Enable the New Relic integration in WayScript.
Set up a New Relic APM agent in your Lair.

1. Open a WayScript workspace for your team

In WayScript, each team gets access to a WayScript workspace. SRE and DevOps teams preconfigure integrations and set organizational standards for the workspace. Developers in turn rapidly create Lairs in the workspace, the basic building blocks of WayScript.

Access WayScript at app.wayscript.com or download the WayScript desktop app available for MacOS and Windows at wayscript.com/downloads.
Create your account by following the sign up instructions.

Create a workspace. Your WayScript workspace is where you and your team will collaborate on managing and shipping software. Note that you can invite others on your team to the workspace by selecting + Invite Team Member at the bottom left of the screen.

2. Create your first Lair in WayScript: a containerized dev environment

A Lair is a preconfigured, containerized, and flexible development environment for your code. They come stocked with handy tools like a cron scheduler, terminal, IDE and endpoint configuration.

To create a Lair in your workspace:

Select the Lair tab in the left panel to navigate back to the Lair view.
In the Lair view, select + New Lair in the top right of the screen.
To get started coding from scratch, select a pre-built template to customize or create a blank Lair. Follow this WayScript guide for inspiration and simple examples. For now, you can select the Flask template under Frameworks.

3. Enable the New Relic integration in WayScript

Note: To ensure your data moves seamlessly from WayScript to New Relic, make sure to complete the integration by following these steps before you deploy a Lair, which is covered in the next section.

Log in to New Relic or sign up for a free account.
Create a New Relic integration in WayScript. In WayScript, go to integrations by selecting the Workspace dropdown menu in the top left corner. Select Settings, and then Integrations. On the integrations page, select +New Integration, and then New Relic.

Monitor Self-Serve Developer Environments with WayScript | New Relic

Enter your New Relic account_id and license_key.

You can find these values on the WayScript data source page in New Relic.
Optionally, you can choose to use a new license key. If so, read the New Relic API key documentation for instructions.

In WayScript, complete the integration by selecting Create integration.

4. Set up APM for your Lairs (Python example)

Next, install a New Relic APM agent to monitor your Lair and anything you build inside it. WayScript supports development in any language, but this example uses the Python agent. To see all the APM agents available in New Relic, go to Add data, and filter for Application monitoring.

Configure the APM agent in the WayScript terminal. Select the Lair you want to monitor and select the Develop tab in the left panel. In the terminal that opens in the right panel, run these commands to generate the newrelic.ini config file for the Lair:

pip install newrelic 
newrelic-admin generate-config YOUR_LICENSE_KEY newrelic.ini

Add New Relic to your requirements.txt file. Open the code editor by selecting the Show Code icon in the bottom left pane. Select the requirements.txt file and add newrelic at the bottom of the file:

If your Lair does not have a requirements.txt file, you can create one directly in the code editor.

Add a deploy trigger. Open the Triggers panel by clicking the last icon, Show Triggers, in the bottom left pane. Click the + icon at the top of the Triggers panel and select deploy. Paste the following command in Command to Run:

NEW_RELIC_CONFIG_FILE=newrelic.ini newrelic-admin run-program $YOUR_COMMAND_OPTIONS

Be sure to replace $YOUR_COMMAND_OPTIONS with your app’s command line, for example, python app.py.

Test the connection. Invoke a process in your Lair. (Select the play icon under Run on the deploy trigger or navigate to the Lair endpoint.)
Deploy your Lair to productions. Select Deploy from the left panel to open the deploy view and click Deploy. This promotes your changes from your development environment to an identical production environment with unique endpoints and will automatically run your deploy trigger.

Wait a few minutes for New Relic to start receiving your data. It should take only 5-10 minutes.

Monitor Self-Serve Developer Environments with WayScript | New Relic

That’s it! Now you should be able to see your Lair and applications in New Relic.

To read this full New Relic blog, click here.

Building for sustainability: Dashboards

Louis Leung — Thu, 29 Feb 2024 22:52:16 +0000

As the software industry increases its focus on environmental, social, and wider sustainability practices, there’s been a rise in new reporting regulations. Regulations like the EU’s Due Diligence proposal put the onus on large and medium companies to identify, prevent, or mitigate damaging environmental and human rights in their practices. The EU’s Taxonomy for Sustainable Activities identifies six environmental objectives for EU-based companies to focus on. Work is now underway to create new metrics and key performance indicators (KPIs) to monitor these actions and communicate them publicly for transparency.

Build or buy?

To fully track your data across your entire supply chain, you'll need to build a dashboard that utilizes data visualization and your regulatory reporting data. With this information in a single interface, you can monitor your carbon footprint and perform benchmarking tests to meet your targets. To implement a dashboard into your tech stack, you have two options: build your own or buy one off the shelf.

Building a dashboard increases your initial workload and lengthens the time to deployment, but it also lets you customize data taxonomies and categories to meet your company’s exact regulatory needs. Here are four steps to start with:

Get access to ESG data and supplier databases.
Invest in a monthly or yearly subscription to license the ESG data set. The license cost varies depending upon the number of companies included in the dataset and the length of time covered by the data.
Connect the databases to a central dashboard.
Categorize your suppliers’ data to fit into your reporting metrics.

Companies can also buy off-the-shelf solutions including pre-made ESG dashboards with taxonomy definitions and category matching for supplier data in place, with options to address different regulatory standards.

Common features in ESG dashboards

Features you’ll find in most ESG dashboards include:

Automated data collection and aggregation.
Portfolio-level insights.
Benchmarking against peers.
Compliance with the EU’s Sustainable Finance Strategy.
Sustainability taxonomy and the Sustainable Finance
Disclosure Regulation (SFDR).
Sustainable Development Goals (SDG) setting and tracking.
Many ESG dashboards do not share data sources, but two datasets are regularly referenced: REFINITIV and CSRHub.

Many dashboard providers also have the following privacy tools and methods to make sure supply chain data remains commercial in confidence and to prevent attacks:

2-factor authentication.
Dedicated client servers.
Admin-controlled whitelisted IPs.
Infrastructure and application vulnerabilities monitoring.
Monitoring threats with a managed security service provider (MSSP).
ISO 27001 compliance certification.
Consumer data protection laws on where dashboard providers are headquartered.

7 important metrics

As new regulations require companies to report on their environmental and social impact, ESG dashboards will become a common tool. To ensure that reporting meets regulatory compliance, data needs to be tracked. Monitoring metrics fall into three categories: integration ops, data ops, and machine learning ops.

1. CPU and memory usage

What it is: The amount of processing power used by your dashboard’s API server to run or consume APIs used in your dashboard.
How to measure it: CPU and memory usage can be measured by installing a cluster in the server that hosts your API code.

2. API consumption

What it is: The volume of requests your APIs receive per second/minute.
How to measure it: Application monitoring tools like New Relic’s APM track API consumption. To lessen the strain on your dashboard APIs, consider combining multiple API calls into a single call with a flexible pagination scheme.

3. Response time

What it is: The time elapsed between when an API request is made and when it was completed. For example, the amount of time between selecting a list of SDGs in your ESG dashboard and that list loading.
How to measure it: Accurate response time can be difficult to measure since latency issues can stem from multiple locations (API endpoints or the network itself, for example).

4. Data volume

What it is: The amount of data you input into the ESG dashboard’s database. By keeping track of data volume, you ensure that all your inputted data has successfully integrated into the database.
How to measure it: Compare the number of new items sent to the database against the number of new items shown in the database—the two numbers should be equal.

5. Corrupted data

What it is: Data errors that occur during transfer or processing. A data error could result in an empty value in your database, which could make future calculations inaccurate if not fixed.
How to measure it: Your combined total of alerts that have been issued with empty or null responses.

6. Data invariance

What it is: The dashboard’s ability to recognize when data doesn’t adhere to a predetermined schema. For example, an inputted data set that doesn’t match up to the sustainability taxonomy it has been trained to recognize.
How to measure it: Your management system should issue an alert if data is incompatible with recognized schemas.

7. Training and serving features

What it is: Tests that determine if the data output in your training environment (where the machine learns data models that form the basis of its future calculations) matches the output of your serving environment (where the calculations take place). For example, if you’ve trained your ESG dashboard to calculate weighted metrics where the sustainability reputation of a company in your supply chain affects another, such as a seller and buyer, those prediction values need to be calculated consistently and accurately.
How to measure it: Calculate distribution statistics, like minimum and maximum values, on the training and serving environments to ensure they match.

To read this full New Relic blog, click here.

Explore new customization options for New Relic dashboards

Louis Leung — Thu, 29 Feb 2024 22:38:18 +0000

New Relic dashboards let you analyze telemetry data across your entire stack, regardless of the source. Dashboards are a flexible way for engineering teams to visualize data however they want by combining pre-made templates, native platform visualizations, custom queries, and more—all in one place.

Now you can enhance your dashboards with even more customization choices, including the ability to:

Tailor your chart colors to highlight key information, identify errors faster, or match your preferred color palette.
Remove or transform null values.
Increase chart detail and clarity with customizable units.
Analyze different time ranges on the same dashboard.
Use Markdown to add custom content to your dashboards.

In this blog, you’ll see examples and tips to help you make the most of your custom dashboards with the latest dashboard options. You’ll also get a sample dashboard JSON file so you can try it out yourself.

Getting started

Prerequisites

Follow along by starting with these steps:

Log into your New Relic account. If you don’t have one, you can sign up for free.
Connect an application or service in New Relic. This ensures that the dashboard will be populated with transaction data. If you haven’t done this before, go to Add data in New Relic and select guided install, or choose an application monitoring data source and follow the installation instructions.

Get the dashboard template

Once you have logged into your New Relic account and connected your application or service successfully, you can get a copy of the dashboard template. To do this, simply go to dashboards in New Relic, then select Import dashboard.

Next, paste the JSON code from this GitHub gist.

Note: Before moving on to the next step, update the account number in the JSON code.

There should be a total of nine instances of the following line: "accountId":

Replace the placeholder with your account number. Here's an example: "accountId": 1231234

If you don’t know your account ID, follow this documentation.

Then, select Import dashboard. You should see something like this.

Color customization

Adding colors to your data visualizations doesn't just improve their aesthetics. Colors can also help convey meaning and draw your eye to critical information. Now you can change the color of your time series charts, on a per-series basis, for area, bar, line, pie, and stacked bar charts.

For example, you might want to select a classic red and yellow scheme for severe or moderate error warnings. Charts that are faceted by country code could also use readily identifiable colors to help readers parse information without needing to look at the chart legend. Or you could simply choose your branding colors for a polished look.

To customize series colors:

Run your query.
Open the Colors menu in the chart settings.
Select the + icon.
Select the series you want to recolor.
Select the desired color. You can select colors graphically or enter a custom RGB, HEX, or HSL code.

Null value settings

Sometimes time series charts can look broken when there is a zero in the event attribute you are querying. However, a null value may not represent the situation correctly. Now you have various options for displaying null values, making your time series charts easier to understand.

The new options for handling null values include:

Leave as null (default): Null values show as gaps in the chart, which can result in disconnected lines. You may want to use this option if it's important to find gaps in reporting, or when it doesn’t make sense to connect a continuous line.
Remove the value: Null points won’t exist in the chart. For example, a chart that displays a data point per hour in a day would show fewer than 24 points.
Preserve the last value: Null values will keep the value from the previous period. For example, if a chart reports cumulative sales each hour, but is missing an hour’s data, the missing hour would reflect the same number of sales as the previous hour.
Transform the value to zero: The missing data shows up as a zero. This option could be useful when no data means there is no meaningful transaction, count, or usage.

To customize how your null values appear:

Select Edit chart or run a query.
Open the Null values menu in the customizations UI.
Select the Null values option that you want to apply.
Optionally, if you’d like to use different options per series, you can select the + button beside Null values per series to configure them.

Unit customization

To provide more clarity on your charts, you can now customize the unit on your Y-axis and in each of your series. It’s available in line, area, and stacked bar charts.

To customize the unit on your Y-axis:

Run a query.
Open the units menu in the customizations UI.
Select the unit that you’d like to use. This will be shown on the Y-axis.
Optionally, use the + icon to add a per series customization. Select the series that needs another unit. These will be shown when you hover over the values.

Ignore the time picker for persistent values

The time picker defines the time range of your analysis. Adjusting the time range at the top of your dashboard sets the same range for all of your charts. This is handy while troubleshooting incidents when you need to narrow down your data to observe what happened in a specific time period. In addition, dashboards make it easy to correlate information across your charts, as you can click and drag within a time series chart to adjust the time picker for the dashboard.

However, there are instances when you might not want a dynamic chart. For example, you might want values to be persistent to show a cumulative count for a specific time period. Or maybe you always want to show the values from the last 24 hours, regardless of the time range the rest of the dashboard shows.

To make a chart and ignore the time picker:

Select Edit chart or run a query.
Open the Dashboard Options menu in the customizations UI.
Turn the toggle on for Ignore time picker.

Markdown widget

Add context and flair to your dashboard with custom content using our Markdown editor. With this feature, you can add formatted text, checklists, images, code snippets, and more. This makes it easy to explain shared dashboards, add useful hyperlinks, create task lists, and create any other Markdown content you need!

To add a Markdown widget:

Select the + symbol in the top right corner of your dashboard.
Select Add text, images, or links.
Enter your inputs into the Markdown pane on the left.
See your content in the Preview pane.
Select Save to add it to your dashboard.
If needed, edit existing content by selecting the ellipses (...) icon on any Markdown widget and selecting Edit.

For more information about Markdown syntax options, see the GitHub Markdown guide.

Check out the video below to learn more about the widget.

To read the full New Relic blog, click here.

Monitor OpenAI GPT application usage in New Relic

Louis Leung — Thu, 22 Feb 2024 01:32:46 +0000

To read this full New Relic blog, click here.

If you're already using or plan to use OpenAI's GPT large language models like GPT-3 and GPT-4 at scale, it's important to monitor metrics like average request time, total requests, and total cost. Doing so can help you ensure that OpenAI GPT Series APIs like ChatGPT are working as expected, especially when those services are required for important functions like customer service and support.

In this post, you’ll learn how you can easily set up the New Relic OpenAI integration, and:

Monitor OpenAI usage and track costs.
Analyze and optimize model performance.
Understand user engagement.

Start monitoring OpenAI in minutes

New Relic is focused on delivering valuable AI and machine learning (ML) tools that provide in-depth monitoring insights and integrate with your current technology stack. Our industry-first MLOps integration with OpenAI’s GPT-3, GPT-3.5, and GPT-4 provides a seamless path for monitoring this service. Our lightweight library helps you monitor OpenAI completion queries and simultaneously records useful statistics around ChatGPT in a New Relic dashboard about your requests.

In this video, see how you can start monitoring OpenAI with just two lines of code and get a pre-built dashboard to help you achieve better GPT model performance.

With just two lines of code, simply import the monitor module from the nr_openai_monitor library and automatically generate a dashboard that displays a variety of key GPT performance metrics such as cost, requests, average response time, and average tokens per request.

To get started, install the OpenAI Observability quickstart from New Relic Instant Observability. Watch the Data Bytes video or read our OpenAI integration documentation for details on how to integrate New Relic with your GPT apps and deploy the custom dashboard.

Track and allocate costs based on token usage

Usage of the OpenAI API is charged based on token consumption, with varying costs across different models. Using OpenAI’s most powerful Davinci model costs $0.12 per 1000 tokens (equivalent to approximately 750 words), which can add up quickly and make it difficult to operate at scale.

Naturally, one of the most valuable metrics you’ll want to monitor is the cost of operating ChatGPT. The New Relic integration for OpenAI provides real-time cost tracking and token consumption. The pre-built dashboard surfaces the financial implications of your OpenAI usage and helps you determine more efficient use cases.

Analyze and optimize model performance

The speed of your ChatGPT, Whisper API, and other GPT requests can help you improve your models and quickly deliver the value behind your OpenAI applications to your customers. Our integration helps you monitor the performance of the OpenAI API by showing average response times. With the pre-built dashboard, you can analyze the aggregate response time metric over time, or see a breakdown of response times by model. With visibility into your model performance, you can understand your usage, troubleshoot faster, and improve the efficiency of ML models.

Understand user engagement

Other metrics included on the New Relic dashboard are total requests, average token/request, model names, and samples. These metrics provide valuable information about the usage and effectiveness of ChatGPT and OpenAI, and they can help you enhance performance around your GPT use cases.

With this integration, you can even gain visibility into user prompts and completions. This allows you to better understand user engagement with your applications to optimize your parameters and settings.

Overall, our OpenAI integration is fast, easy to use, and will get you access to real-time metrics that can help you optimize your usage, enhance ML models, reduce costs, and achieve better performance with your GPT models.

To read this full New Relic blog, click here.

Get faster and more secure software delivery with JFrog and New Relic

Louis Leung — Thu, 22 Feb 2024 01:23:07 +0000

To read this full New Relic blog, click here.

JFrog is used by millions of developers and DevSecOps teams to manage and control their software development lifecycle. With solutions like JFrog Artifactory, the industry-standard artifact repository manager, and JFrog Xray, you can maintain high levels of security and reliability while improving compliance of your entire software supply chain from code to production.

But what if you could optimize your software delivery processes, improve software quality, reduce risk, and gain a competitive advantage by visualizing and analyzing the performance of your software supply chain in real-time? What about getting full observability into all your JFrog applications, artifacts, and dependencies—including metrics related to build times, artifact downloads, and deployment success rate along with it?

That’s why we’ve partnered with JFrog to launch a new quickstart integration, providing DevOps, DevSecOps, and engineers full visibility into the performance and security of your software releases from your JFrog environments to quickly identify potential bottlenecks and areas for improvement. The integration is available in New Relic Instant Observability, the largest open ecosystem of integrations, tools, and pre-built resources designed to help engineers embrace observability as a daily practice.

More Secure Delivery with our JFrog Integration | New Relic

All full New Relic users and all tiers of self-hosted JFrog customers can access it for free. Benefits of this integration include:

Faster and more secure software delivery. Monitor all your JFrog applications, artifacts, and dependencies in real time with pre-built dashboards to correlate performance alongside the rest of your telemetry data in a unified view.
Better compliance and security. Identify vulnerabilities and malicious users with JFrog Xray scanning tools, then prioritize security issues with New Relic vulnerability management to deliver more secure software with less toil.
Improved software supply chain performance. Gain insights into the various stages of the software delivery pipeline, such as build, test, and, deployment, to proactively manage performance degradation, remediate issues, and boost release velocity and quality.

Get real-time insights of your software supply chain in minutes

In this blog post, you’ll learn how to connect and visualize data from JFrog and get full observability into all your applications, artifacts, and dependencies. By the end of this post, you’ll be able to:

Set up the integration to send JFrog log data and key metrics to New Relic
Understand service performance and utilization with the Artifactory dashboard
Mitigate vulnerabilities with Xray dashboard in New Relic

To get started, you'll need to connect logs and metrics data from JFrog to New Relic. To send logs, this integration uses Fluentd, a lightweight open source data collector that unifies log collection and consumption. It also uses OpenMetrics to transmit key metrics. For full installation instructions of the New Relic Fluentd plugin to forward logs, see the readme.

Install the quickstart for instant JFrog observability

Once you have Fluentd running, the best way to start monitoring your JFrog environments is with one of the pre-built dashboards. Once you have JFrog data flowing into New Relic, you can add our quickstart dashboards to get an overview of your environments in less than a minute:

Go to the JFrog quickstart in New Relic Instant Observability, and select Install now.
Select an account and select Begin installation.
If you've already completed the earlier integration steps in this tutorial, select Done to move on to the next step.
The quickstart deploys the resources to your account. Select See your data, and then select the dashboard you’d like to see.

Improve decision-making with actionable insights with the JFrog Artifactory dashboard

The JFrog Artifactory dashboard provides real-time visibility into the performance, health, and usage of your environments. You’ll get answers to questions like:

"What is my most requested artifact?”
“What services are running slow?”
“Who are my heaviest users?”
With these answers, you can accurately provision resources, fix errors, and update permissions.

For example, you might notice a spike in errors. You can investigate whether it could be coming from a known user that is misconfigured and needs additional permissions. Or, perhaps you’ll discover from the performance metrics that an instance is overloaded.

More Secure Delivery with our JFrog Integration | New Relic

Overview page

Start here to get a snapshot of your artifactory services. Each widget is labeled with the relevant dashboard page where you can learn more.

Get key metrics such as disk and memory usage, error rates, and heap memory.
See a breakdown of top HTTP response failures and data transfer by repo.
You can even dive into individual log details with log management in the connected UI.

System Metrics page

Track the performance of your artifactory service.

See usage for disk space, CPU, memory, heap memory, and processors over time.
Application page

Track log volume and artifactory errors over time.

Get error rates, log volume, and HTTP connections.
Audit page

Track audit logs to determine who is accessing your artifactory instance and from where.

Get a breakdown of denied actions and logins by username and IP.
See which users have the most accepted deployments.
Use the dashboard variables to quickly filter by user and IP address.
Docker page

Track your Docker Hub status over time.

Get a breakdown of uploads and downloads by repo.
See which images and repos are being accessed.
Use variables to filter your charts by repo.
Database page

Track key database metrics such as active connections, idle connections, and DB connection utilization.

Requests page

Track HTTP response codes as well as upload/download activity by IP address.

Uncover and mitigate vulnerabilities with the JFrog XRay dashboard

For DevSecOps, it’s important to understand your security posture and to know where a vulnerable build has been deployed into production. New Relic Vulnerability Management already helps you reduce the vulnerability surface area of your software systems, and now, with the JFrog XRay dashboard, you can also:

Discover and track violation trends to monitor for unwanted increases in security threats in your software supply chain.
Drive continuous security by tracking the most downloaded artifacts and surfacing the artifacts which harbor the most common vulnerabilities and exposures (CVEs).
Discover bad users and user misuse by monitoring who is accessing your artifacts and from what IPs.

The dashboard is divided into three pages:

Violations page: See an aggregated summary of all the license violations and security vulnerabilities found by Xray.

Breakdown of violations by watch, severity, policy, and rules.
See top impacted artifacts, infected components, and CVEs.
See the most downloaded vulnerable artifacts.
Get violation details down to the signature.
Logs page: View summaries of access, service, and traffic log volumes associated with Xray. Track various HTTP response codes, HTTP 500 errors, and log errors for greater operational insight.

Metrics page: See system performance, storage consumption, connection statistics, and the count and type of artifacts and components that are scanned by JFrog Xray.

View even more JFrog metrics and logs

While the quickstart gives you just about everything you need to begin monitoring JFrog in New Relic, you can also manually query, display, and create alerts to dive into individual metrics and get full control over your JFrog monitoring strategy.

To see available metrics, select Query your data > Data explorer > Metrics, and search for jfrog. You can use these metrics in the New Relic query builder or your NRQL queries.

You can use these metrics in the New Relic query builder or your NRQL queries.

To explore JFrog logs, go to Logs in New Relic, then search for log_source:"jfrog". You can select an individual log summary for more details as shown in this screenshot:

This should get you started with JFrog Software Supply Chain Platform and New Relic to get the insights you need to boost release velocity, security, and quality. Keep an eye out for more integrations between JFrog and New Relic coming soon!

To read this full New Relic blog, click here.

Unlock the power of observability for marketing leaders

Louis Leung — Thu, 22 Feb 2024 01:14:51 +0000

To read this full New Relic blog, click here.

As a marketer, data is one of the most crucial keys to growing the business. Software engineers are known for working with vast amounts of telemetry data to monitor, troubleshoot, and improve the performance of their applications and systems. But what if I told you that the powerful observability tools they use can also be a marketer’s secret weapon?

In this blog post, you'll see how marketing leaders can use the New Relic observability platform to improve their website performance, track marketing campaigns, and monitor the health of their backend.

Unlock the power of observability for marketing leaders | New Relic

Benefits of using New Relic for marketing and business leaders include:

Increased visibility: New Relic provides a single pane of glass for monitoring your entire IT infrastructure, giving you a comprehensive view of your performance and health.

Early detection: New Relic alerts and notifications can help you detect performance issues early, before they impact your users.

Proactive troubleshooting: New Relic data can help you identify the root cause of performance issues so that you can fix them quickly and easily.

Improved decision-making: Utilize the data from all your systems, applications, and websites to make better decisions about allocating your resources and improving your business performance.

Easily understand and improve site performance

Unlock the power of observability for marketing leaders | New Relic

Speedy sites attract more users and revenue. The New Relic website performance monitoring tool allows you to track the performance of your (or anyone else’s) website without manual setup. By simply entering a URL, you get a dashboard to continuously monitor key performance metrics that search engines care about, such as Google Core Web Vitals. This data can be used to identify and fix performance issues, improve the user experience, and benchmark with other web pages.

Built-in capabilities include:

Historical and ongoing reporting, unlike other one-off manual site checkers.
Automated alerts for page down and broken links.
Scheduled website checks, as often as you need.
Recommendations for best practices and how to troubleshoot issues.

Track conversions, campaigns, and launches

Logs unlock a deeper level of analysis, allowing you to track the performance of your marketing campaigns. With this data, you can get a near-real-time view of every user interaction on your website. That means you’ll be able to answer questions like:

Which campaigns are driving results?
Which sources and mediums are working best?
Who are my top referrals and lead sources?

As a result, you make better decisions on where to spend your marketing dollars.

More importantly, unlike with other tools, you no longer have to wait hours or even days to find out that something is broken. You can proactively steer your business in the right direction.

Monitor backend health

The health and performance of your servers and hosts are critical for a great customer experience. New Relic provides visibility into the backend by tracking metrics such as throughput, web transactions, and errors. This data can be used to identify and fix performance issues, prevent bot attacks, and ensure that your website is always up and running.

See how New Relic provides a full view of your system’s backend in a single place in the video below.

Unlock the power of observability for marketing leaders | New Relic

To read this full New Relic blog, click here.

Integrations to monitor your full AI stack

Louis Leung — Wed, 13 Dec 2023 17:30:10 +0000

Quickstart integrations for your LLMs, vector databases, frameworks, and more. To read this full blog, click here.

As organizations integrate AI into their tech stacks to improve efficiency and provide better customer experiences, a variety of tools are emerging for developing and deploying AI applications. However, bringing these applications from prototype to production is not without challenges. AI components like large language models and vector databases, while powerful, can be opaque and may lead to issues like inaccurate and biased results, security vulnerabilities, and new types and overwhelming amounts of telemetry data for analysis.

To help solve this problem, we announced New Relic AI monitoring (AIM). This innovative solution offers unparalleled end-to-end visibility into your AI-powered applications, empowering you to optimize performance, quality, and cost in your AI application development. With New Relic AI Monitoring, adopting AI becomes easier and provides confidence in harnessing the full potential of AI in driving business growth and efficiency.

To extend observability into specialized tools and technologies, we have developed over 50 quickstart integrations to help provide out-of-the-box visibility into each layer of your AI tech stack, including:

Foundational models (for example, large language models or LLMs)
Orchestration frameworks
ML libraries
Model serving
Storage and registries
Infrastructure

Orchestration framework: LangChain

Orchestration frameworks like LangChain allow developers to chain together different components of an AI application, such as data processing, model invocation, and post-processing. This makes it easier to build and deploy AI applications that are modular, extensible, and scalable.

LangChain is a popular orchestration framework because of its flexibility and ease of use. It provides a library of pre-built components that can be combined to create custom AI applications.

The New Relic LangChain quickstart integration provides a single view of all components of your AI application, including models, chains, and tools. The pre-built quickstart dashboard visualizes prediction times, token usage, and even breaks down the contents of prompts and responses. This allows you to track performance and throughput, identify bottlenecks, and optimize your application for efficiency. Moreover, the integration can automatically detect different models such as OpenAI and Hugging Face so you can compare and optimize your LLMs.

Foundational AI models: OpenAI, Amazon Bedrock, PaLM 2, Hugging Face

AI models are algorithms that have been trained on data to perform specific tasks, such as recognizing objects in images, translating languages, or generating text. OpenAI offers various popular generative AI models, as well as APIs that you can use to integrate this functionality into your applications. New Relic introduced the industry’s first OpenAI observability integration, which helps you monitor usage, analyze model performance, and optimize costs for your OpenAI applications. By adding just two lines of code, you can gain access to key performance metrics such as cost, requests, response time, and sample inputs and outputs.

Additionally, New Relic is also integrated with Amazon Bedrock, a fully managed service by AWS that makes building and scaling generative AI applications more accessible by providing API access to foundation models from leading AI companies, including AI21 Labs, Anthropic, Cohere, and Stability AI. Now with the Amazon Bedrock integration, you can easily monitor performance and usage of the Amazon Bedrock API and its connected LLMs.

Furthermore, if you chain multiple models to LangChain, you can compare the performance of LLMs within a single pre-built dashboard. This includes integrations with popular models and model hosting platforms such as PaLM 2 and Hugging Face.

ML libraries: PyTorch, Keras, TensorFlow, scikit-learn

Machine learning libraries are an essential component of the AI stack. They provide the tools and libraries that developers need to build and train AI models. New Relic has integrated with popular ML libraries such as PyTorch, Keras, TensorFlow, and scikit-learn to tackle model degradation by monitoring performance and health metrics such as inference latency, memory usage, and data drift. This allows for early detection and helps you identify when to optimize your ML libraries.

Model serving: Amazon SageMaker, Azure Machine Learning

The model serving and deployment layer is responsible for making AI models available to users. It is responsible for loading and running AI models, as well as providing an API or other interface for users to interact with the models. Handling the complex and resource-intensive process of developing AI applications requires a centralized platform for managing the entire AI development lifecycle, from design to deployment.

Popular AI platforms that offer this functionality include Amazon SageMaker and Azure Machine Learning. The Amazon Sagemaker and Azure Machine Learning quickstart integrations provide visibility into training jobs, endpoint invocations, and operational efficiency with pre-built dashboards and alerts. By monitoring key metrics like job executions, endpoint deployments, and CPU and GPU usage, you can ensure that your AI projects can adequately be supported by your infrastructure while troubleshooting user experience.

Learn more about the Azure Machine Learning integration by reading this blog post.

Storage and registries: Pinecone, Weaviate, Milvus, FAISS

AI applications need to store and access large amounts of data. Vector databases are specialized databases that are designed to store and query high-dimensional data more efficiently with similarity search rather than exact matches. They’re often used for AI applications because they can reduce the cost and increase the speed of training and deploying AI models.

To help monitor database health, New Relic provides pre-built quickstart dashboards for various data storage and registry solutions such as Pinecone, Weaviate, Milvus, and FAISS. In addition to key database metrics such as execution and response latency, requests, and disk space usage, they also allow you to track metrics specific to vector databases such as indexing performance. You can connect your data via chains in LangChain or through Pinecone’s Prometheus endpoint.

Infrastructure: AWS, Azure, Google Cloud Platform, Kubernetes

Integrations to Monitor Your AI Stack | New Relic

New Relic offers a wide range of infrastructure monitoring solutions and integrations including:

Our infrastructure agent for Linux, macOS, and Windows operating systems.
Our Kubernetes and Pixie integrations.
Our cloud integrations for Amazon, Azure, and Google Cloud Platform.
Our Prometheus integrations.
Our on-host integrations for services like Apache Spark, MySQL, NGINX, Cassandra, and Kafka (to name a few). Most of the on-host integrations are installed with the infrastructure agent and work in concert with it to send data.

Working with the latest AI and machine learning technologies allows you to build amazing experiences. But like any other software experience, your monitoring strategy is key to keeping your user experiences running smoothly. With our integrations for AI and model performance monitoring, New Relic helps you gain visibility into the performance of every layer of your AI stack to easily identify performance, cost, and quality issues affecting AI applications.

To read this full blog, click here.

New Relic launches API monitoring with Postman integration

Louis Leung — Fri, 08 Apr 2022 22:52:55 +0000

More than 17 million developers rely on the Postman API platform to design, build, test, document, deploy, and monitor APIs within a collaborative platform. Postman’s monitoring feature allows users to observe API performance and quality in detail, but downstream operations teams may only get a high-level view of API performance with metrics such as CPU utilization, memory consumption, and database throughput. Similarly, for developers to understand how a single API affects the overall application within a broader deployment, Postman needs to ingest contextual data for APIs from another telemetry source.

Introducing New Relic API monitoring with Postman

To solve these problems, New Relic and Postman have built an end-to-end integration so you can monitor the functionality and performance of your APIs and Postman monitors alongside the rest of your observability data. Now, you can get deeper pre-production API metrics down to the individual endpoint level. You can also use the two-way integration to receive contextual data about deployments from New Relic One in the Postman API platform.

The Postman and New Relic integration brings API observability data to developers and engineers where they already spend their time—whether they are DevOps engineers monitoring their stack in New Relic One or API developers seeking real-time application performance data in context of their API deployments in Postman.

To make it even easier to view your API data, we've also created a new quickstart in New Relic Instant Observability (I/O) that will help you get up and running faster with a pre-built dashboard that includes the API and infrastructure metrics you need in one place.

Here's an example of the Postman monitor dashboard in New Relic:

All developers can get started for free with New Relic and use the API monitoring feature as part of the more than 450 integrations included with New Relic's observability platform. Every new sign-up in New Relic receives 100GBs per month of free data ingest, one full platform user, and unlimited basic users, queries, dashboards, and alerts.

Benefits of the new integration and quickstart include:

Unified visibility: View API metrics (response time, errors, failed test) alongside application and infrastructure metrics in one unified and customizable dashboard.
Create alerts: Leverage New Relic's advanced alerting capability such as self-adjusting thresholds, anomaly detection, and outlier detection so you get notified when your APIs have poor performance.
Debug performance issues in context: Debug performance issues by comparing the performance of your APIs (such as latency metrics) and functionality (such as response correctness) with full stack metrics (CPU, RAM usage, and network stats) in context.
Easy access: Existing customers with New Relic full platform access can set up the integration for no additional cost. New customers can get started for free without a credit card.

Viewing New Relic APM data in Postman

How to send real-time API metrics from Postman to New Relic

To start monitoring your API performance and visualize results from Postman monitors in New Relic, watch the following video or follow the steps listed after the video.

Set up your accounts. You need to have New Relic and Postman accounts to use the integration. If you aren't already using New Relic One, get a free account. (2 minutes)
Retrieve your New Relic API key. Go to your API keys in New Relic. Select the three dots next to the INGEST - LICENSE key and select Copy key to save it for later. (1 minute)
Enable the integration in Postman. From Integrations, search for New Relic. There are two integrations available. Select the option labeled Bring Postman monitor results into New Relic and add the integration by using the API key from step 2. Follow the steps in the Configure New Relic Integration documentation. (5 minutes)
Visualize data with the quickstart. After you've enabled Postman metrics to flow to New Relic One, install the quickstart to get a curated dashboard. Go to the Postman quickstart in New Relic I/O, select the + Install quickstart button, and follow the guided installation process. (5 minutes)

After you are finished, you automatically get a curated dashboard where you can see key API metrics from your Postman monitors.

View Postman metrics and dimension attributes

The quickstart gives you everything you need to begin monitoring key API metrics in New Relic One. Optionally, you can also manually query, display, and alert on metrics from Postman. As soon as your monitor runs, Postman starts sending metrics data to New Relic.

Optional: To manually go through the process of viewing your metrics and building queries in New Relic One, select Browse data, then select Metrics and search for postman. You can use these metrics in the New Relic query builder or your NRQL queries.

Several attributes such as monitor.name and request.id can also be used as dimensions. For example, you can group latency data by region:

See a full list of metrics and their descriptions as well as all dimension attributes in the Metrics pushed to New Relic and Dimension attributes sections of the documentation.

Next Steps

Install the Postman quickstart and get started in minutes.
Review the integration documentation and Nerd Byte instructional video.
Check out the Integrate with New Relic in Postman video on how to view New Relic APM data in Postman.
If you're not already using New Relic One, get started with New Relic for free. Your free account includes 100 GB/month of free data ingest, one free full-access user, and unlimited free basic users.
If you're not already using Postman, sign up for Postman. The integration is available to all Postman customers, including free-tier users.

DEV Community: Louis Leung

New Instant Observability integrations for monitoring API, network, application, and streaming performance

New partner integrations

Cloudflare Network Logs

Netlify Logs

Postman

Datazoom

Glassbox

Get your insights in minutes

The democratization of observability

A single ecosystem for integrations, applications, and quickstarts

How to install your first quickstart

Partners in instant observability

Partners in Instant Observability: Integrations for machine learning, Kubernetes, CI/CD

New partner quickstarts

Instant observability for CI/CD

Instant observability for Kubernetes testing

Instant observability for AI/ML model monitoring

Bring your own ML data into New Relic One

Automate root cause analysis in logs with New Relic and Zebrium

To read this full New Relic blog, click here.

Identify root causes at a glance with the Zebrium integration

How to get started with automated root cause identification in minutes

Create and monitor WayScript self-service dev environments in minutes

Automate development environment monitoring with New Relic and WayScript

How to create and monitor your environment in minutes

1. Open a WayScript workspace for your team

2. Create your first Lair in WayScript: a containerized dev environment

3. Enable the New Relic integration in WayScript

4. Set up APM for your Lairs (Python example)

Building for sustainability: Dashboards

Build or buy?

Common features in ESG dashboards

7 important metrics

1. CPU and memory usage

2. API consumption

3. Response time

4. Data volume

5. Corrupted data

6. Data invariance

7. Training and serving features

Explore new customization options for New Relic dashboards

Getting started

Prerequisites

Get the dashboard template

Color customization

Unit customization

Ignore the time picker for persistent values

Monitor OpenAI GPT application usage in New Relic

Start monitoring OpenAI in minutes

Track and allocate costs based on token usage

Analyze and optimize model performance

Understand user engagement

Get faster and more secure software delivery with JFrog and New Relic

Unlock the power of observability for marketing leaders

Easily understand and improve site performance

Track conversions, campaigns, and launches

Monitor backend health

Integrations to monitor your full AI stack

Orchestration framework: LangChain

Foundational AI models: OpenAI, Amazon Bedrock, PaLM 2, Hugging Face

ML libraries: PyTorch, Keras, TensorFlow, scikit-​​learn

Model serving: Amazon SageMaker, Azure Machine Learning

Storage and registries: Pinecone, Weaviate, Milvus, FAISS

Infrastructure: AWS, Azure, Google Cloud Platform, Kubernetes

New Relic launches API monitoring with Postman integration

Introducing New Relic API monitoring with Postman

How to send real-time API metrics from Postman to New Relic

View Postman metrics and dimension attributes

Next Steps

ML libraries: PyTorch, Keras, TensorFlow, scikit-learn