Simple Schema for Observability

Introduced 2.6

Observability is a collection of plugins and applications that let you visualize data-driven events by using Piped Processing Language (PPL) to explore and query data stored in OpenSearch. Simple Schema for Observability, which uses the schema convention ss4o, is a standardization for conforming to a common and unified observability schema. With the schema in place, Observability tools can ingest, automatically extract, and aggregate data and create custom dashboards, making it easier to understand the system at a higher level.

The Simple Schema for Observability is inspired by both OpenTelemetry and the Elastic Common Schema (ECS) and uses Amazon Elastic Container Service (Amazon ECS) event logs and OpenTelemetry (OTel) metadata.

Alerts will be supported in a future release.

Use cases

Use cases for the Simple Schema for Observability include:

• Ingesting observability data from different data types.
• Moving from proprietary configurations that are non-transferable to a consolidated, sharable observability solution that allows users to ingest and display an analysis of any type of telemetry data from any type of provider.
• Conforming dashboards to the schema to align with the data structure so that you can design and organize the dashboard components and visualizations in a way that effectively represents your data.

Data Prepper conforms to the schema for metrics and will gradually support traces and logs. Data Prepper’s trace mapping currently provides service-map data in a different way than ss4o traces. To make the trace mapping compatible with Observability, it will be integrated with the ss4o traces schema and will introduce service-map as an enriched field.

Traces and metrics

Schema definitions for traces and metrics are defined and supported by the Observability plugin. These schema definitions include:

• The index structure (mapping).
• The index naming conventions.
• A JSON schema for enforcement and validation of the structure.
• The integration feature for adding preconfigured dashboards and assets.