A few users reported mapping errors on a few integrations. We suspect these problems may be related to integrations that migrated to ecs@mappings with recent updates.

Here is the list of fields with mapping issues:

Root Causes

ecs@mappings+type-coercion

The ecs@mappings component template does not perform type coercion, so if the value is a string, ES maps it as a keyword.

Here is an example, if I perform the following requests using the Dev Tools:

DELETE _data_stream/logs-whatever-sdh5075
POST logs-whatever-sdh5075/_doc
{
  "@timestamp": "2024-08-20T16:58:01+02:00",
  "destination": {
    "port": "8080"
  }
}
GET logs-whatever-sdh5075/_mapping/field/destination.port

I get the following result:

{
  ".ds-logs-whatever-sdh5075-2024.08.22-000001": {
    "mappings": {
      "destination.port": {
        "full_name": "destination.port",
        "mapping": {
          "port": {
            "type": "keyword",
            "ignore_above": 1024
          }
        }
      }
    }
  }
}

ecs@mappings+date_detection:false

When date_detection is disabled, the following fields aren’t mapped correctly:

threat.indicator.first_seen
threat.indicator.modified_at
threat.enrichments.indicator.modified_at
threat.enrichments.matched.occurred
threat.enrichments.indicator.first_seen 
threat.enrichments.indicator.last_seen
threat.indicator.last_seen 

integration-mapping-problem

We probably need to change mappings in the integration to something similar (like most other integrations do):

- name: client.geo.location
  external: ecs
- name: source.geo.location
  external: ecs

Or remove these mappings and only use ecs@mappings.

integration-update

Mapping changed due to integration updates.