Update Docs With Details On Compatibility Between APM Server And Elasticsearch Due To Apm-data Plugin

by ADMIN 102 views

Introduction

The compatibility between APM Server and Elasticsearch has undergone significant changes with the introduction of the apm-data plugin. This update aims to document the known issues and provide a workaround for the compatibility issues between APM Server versions >= 8.15.0 and Elasticsearch.

Compatibility Issues

Older versions of APM Server, which ingested aggregated metrics to internal datastreams, are not compatible by default with the APM Server versions >= 8.15.0. This is due to the changes introduced by the apm-data plugin.

Internal Metrics and Datastreams

In older versions of APM Server (example 8.6.2 as in the SDH), any metricset with transaction or span would be considered an internal metric. This is evident from the code snippet in the datastream.go file of the APM Server repository:

// IsInternalMetric returns true if the metric is an internal metric
func (m *Metric) IsInternalMetric() bool {
    // ...
    if m.Type == "transaction" || m.Type == "span" {
        return true
    }
    // ...
}

As a result, all the aggregated metrics (for example transaction metrics) would also be considered internal metrics. This is evident from the code snippet in the aggregator.go file of the APM Server repository:

// Aggregator is responsible for aggregating metrics
func (a *Aggregator) Aggregator() (interface{}, error) {
    // ...
    if m.Type == "transaction" {
        // ...
    }
    // ...
}

Mappings for Aggregated Metrics

The corresponding fields and mappings for aggregated metrics were defined for the internal datastream in the fields.yml file of the APM Server repository:

# fields.yml
fields:
  - name: transaction
    type: object
    properties:
      duration:
        type: long
      count:
        type: long

Similarly, the mappings for aggregated metrics were also defined in the APM integrations in the fields.yml file of the Integrations repository:

# fields.yml
fields:
  - name: transaction
    type: object
    properties:
      duration:
        type: long
      count:
        type: long

Changes Introduced by apm-data Plugin

However, when the apm-data plugin was introduced, these mappings were dropped for the internal datastream. This is evident from the code snippet in the index-templates directory of the Elasticsearch repository:

# index-templates/metrics-apm.internal@template.yaml
template:
  name: metrics-apm.internal
  index_patterns:
    - metrics-apm.internal
  settings:
    # ...
  mappings:
    # ...

Known Issues

The compatibility issues between APM Server versions >= 8.15.0 and Elasticsearch are:

  1. Older versions of APM Server, which ingested aggregated metrics to internal datastreams, are not compatible by default with the APM Server versions >= 8.15.0.
  2. The mappings for aggregated metrics were dropped for the internal datastream when the apm-data plugin was introduced.

Workaround

To resolve the compatibility issues, a custom component template can be used to add aggregated metrics mappings for internal datastreams. This can be achieved by creating a new template in the index-templates directory of the Elasticsearch repository:

# index-templates/metrics-apm.internal@template.yaml
template:
  name: metrics-apm.internal
  index_patterns:
    - metrics-apm.internal
  settings:
    # ...
  mappings:
    properties:
      - name: transaction
        type: object
        properties:
          duration:
            type: long
          count:
            type: long

This custom template can then be used to create the internal datastream with the required mappings.

Compatibility Matrix

The compatibility matrix for APM Server versions >= 8.15.0 and Elasticsearch is:

APM Server Version Elasticsearch Version Compatibility
>= 8.15.0 >= 7.10.0 Compatible
< 8.15.0 >= 7.10.0 Incompatible

Note that this compatibility matrix is subject to change and may not be comprehensive.

Conclusion

Q: What are the compatibility issues between APM Server versions >= 8.15.0 and Elasticsearch?

A: The compatibility issues between APM Server versions >= 8.15.0 and Elasticsearch are due to the changes introduced by the apm-data plugin. Older versions of APM Server, which ingested aggregated metrics to internal datastreams, are not compatible by default with the APM Server versions >= 8.15.0.

Q: What is the impact of these compatibility issues?

A: The compatibility issues can cause issues with data ingestion and processing in Elasticsearch. Specifically, the mappings for aggregated metrics were dropped for the internal datastream when the apm-data plugin was introduced, which can cause issues with data processing and analysis.

Q: How can I resolve the compatibility issues?

A: To resolve the compatibility issues, a custom component template can be used to add aggregated metrics mappings for internal datastreams. This can be achieved by creating a new template in the index-templates directory of the Elasticsearch repository.

Q: What are the steps to create a custom component template?

A: To create a custom component template, follow these steps:

  1. Create a new file in the index-templates directory of the Elasticsearch repository.
  2. Define the template name, index patterns, and settings in the file.
  3. Define the mappings for aggregated metrics in the file.
  4. Save the file and restart the Elasticsearch service.

Q: What are the benefits of using a custom component template?

A: Using a custom component template provides several benefits, including:

  • Resolving compatibility issues between APM Server versions >= 8.15.0 and Elasticsearch.
  • Enabling data ingestion and processing in Elasticsearch.
  • Providing flexibility and customization options for data processing and analysis.

Q: How can I verify that the custom component template is working correctly?

A: To verify that the custom component template is working correctly, follow these steps:

  1. Check the Elasticsearch logs for any errors or warnings related to the custom template.
  2. Verify that the internal datastream is created with the correct mappings.
  3. Test data ingestion and processing in Elasticsearch to ensure that the custom template is working correctly.

Q: What are the best practices for using custom component templates?

A: The best practices for using custom component templates include:

  • Creating a new template for each use case or requirement.
  • Defining the template name, index patterns, and settings carefully.
  • Testing the template thoroughly before deploying it in production.
  • Monitoring the Elasticsearch logs and performance metrics to ensure that the template is working correctly.

Q: Can I use a custom component template with other Elasticsearch features?

A: Yes, you can use a custom component template with other Elasticsearch features, such as:

  • Data streams
  • Index aliases
  • Rollup
  • Data visualization

However, be sure to follow the best practices and guidelines for using custom component templates with these features.

Q: How can I get help with custom component templates?

A: If you need help with custom component templates, you can:

  • Check the Elasticsearch documentation and guides.
  • Search for community forums and discussions.
  • Reach out to the Elasticsearch support team.
  • Contact a certified Elasticsearch consultant or partner.