Observability

Read the documentation

Efficiently find and fix problems, improve application health, and deliver better customer experiences

Three foundational observability signals are metrics, logs (semi-structured data), and traces (flows of requests from beginning to end across all dependencies). These signals are the output of monitored environments, like containers, microservices, and applications. The goal is to provide an integrated experience for DevOps and Site Reliability Engineers to isolate critical events and use all the observability signals to isolate issues to containerized applications and microservices running anywhere. Amazon OpenSearch Service combines log and trace data analytics into a single solution.

Observability operations

Amazon OpenSearch Service provides new capabilities to help solve your observability problems.

Features

Use open interfaces to collect, route, and transform telemetry data (including OpenTelemetry, Fluentd, Fluent Bit, Logstash, Data Prepper, and more). You can search and analyze large amounts of semi-structured data with native capabilities. You can visualize, monitor, and alert with anomaly detection observability features of OpenSearch Dashboards, and conduct interactive analysis and visualization on data with Piped Processing Language (PPL), a query interface.

Collect

First, you need to collect data for analysis. Collection includes gathering, enriching, filtering, transforming, and normalizing data from multiple sources.

Detect

Often customers don’t detect issues as soon as they began, there is often a lag from when an issue starts and when you are notified. You want to reduce this as much as possible. Detection should be proactive and multi-faceted (such as alarms on telemetry). Anomaly detection is a key tool, as well as the ability to link together related alarms to reduce alarm fatigue. A core component of detection is also visualization and monitoring, which Amazon OpenSearch Service does with a component called OpenSearch Dashboards. You can even interactively analyze the data with tools like PPL.

Investigate

Investigation is where people spend the most amount of time during an operational event—and the investigation usually takes multiple people. This is the largest contributor to Mean Time to Incident (MTTI) and Mean Time to Recovery (MTTR). Cutting through the chaos and understanding what to focus on remains a difficult task. Use logs, metrics, and tracing to help you quickly conduct root cause analysis—while correlating across metrics, logs, and traces—on AWS, on premises, or on other clouds. Collaborate on the investigations and document your analysis with OpenSearch Dashboard notebooks.

Remediate

After you identify the cause of a failure, you need to remediate it. There is nothing worse than trying to fix something and making the situation worse. Don’t forget to do a post-event analysis to determine how you could have prevented the failure in the first place. Document proposed changes so you can prevent the issue from recurring. Your goal should be to ensure the same issue never happens again—but if it does, that you can identify and remediate it automatically.

Application Performance Monitoring

Sometimes Application Performance Monitoring (APM) is the first maturity level of observability. But APM alone is not enough. Is your application actually performing as expected, even if your application monitoring dashboard is all green? Are your customers getting the user experience they need? What’s the usage of your application? Which parts of your application are hitting scale limits? From which geographic region are you seeing the biggest growth? Which trends can you visualize and plan for? If you could gather metrics, you could have confidence that when you deploy new code or change your infrastructure, you can see the impact of these changes. Observability advances APM to answer these additional questions.

computer displaying performance monitoring

Customers

Chase International: Always-on customer experience at scale
Dow Jones and 3M: Observability with Amazon OpenSearch Service

Observability resources

Displaying 1-8 (11)

Page topics

General
5

General

Trace Analytics is a new feature of Amazon OpenSearch Service that enables developers and IT operators to find and fix performance problems in distributed applications, which leads to faster problem resolution times. Trace Analytics is built using OpenTelemetry, a Cloud Native Computing Foundation (CNCF) project that provides a single set of APIs, libraries, agents, and collector services to capture distributed traces and metrics, which enables customers to leverage Trace Analytics without having to re-instrument their applications. Trace Analytics is powered by the OpenSearch, which is open source and freely available for everyone to download and use.

Developers and IT Ops need Trace Analytics to find and fix performance problems in their distributed applications. By adding trace data to the existing log analytics capabilities of Amazon OpenSearch Service, customers can use the same service to both isolate the source of performance problems and diagnose their root cause. In addition, with the support for the OpenTelemetry standard, Trace Analytics supports integration with Jaeger and Zipkin SDKs, two popular open source distributed tracing systems, which allows developers to continue using these SDKs and not have to re-instrument their applications.

Trace Analytics is an integrated feature of Amazon OpenSearch Service. It is available to all customers at no extra charge. Trace Analytics has a user interface based on OpenSearch Dashboards and Kibana for visualizing and exploring trace data and is integrated with key features of Amazon OpenSearch Service such as anomaly detection, alerting, fine-grained access control, and enterprise security. Trace Analytics complements customers’ usage of Amazon OpenSearch Service for search and analysis of log data when resolving application performance problems.

Trace Analytics today supports the collection of trace data from application libraries and SDKs that are compatible with the open source OpenTelemetry Collector, including Jaeger, Zipkin, and X-Ray SDKs. Trace Analytics also integrates with AWS Distro for OpenTelemetry, which is a distribution of OpenTelemetry APIs, SDKs, and agents/collectors. It is a performant and secure distribution of OpenTelemetry components that has been tested for production use and is supported by AWS. Customers can use AWS Distro for OpenTelemetry to collect traces and metrics for multiple monitoring solutions, including Amazon OpenSearch Service and AWS X-Ray for trace data and Amazon CloudWatch for metrics.

To get started with Trace Analytics, follow the documentation here.