fix(aws/securityhub_findings): event.kind as keyword#14251
fix(aws/securityhub_findings): event.kind as keyword#14251andrewkroh merged 4 commits intoelastic:mainfrom
Conversation
Change event.kind to keyword instead of constant_keyword. When a pipeline error occurs, event.kind was being set to 'pipeline_error' and this break ingestion because event.kind was already assigned a value of 'state'.
Remove unused attributes. Add 'external: ecs' to all ECS fields. [git-generate] go run github.com/andrewkroh/fydler@c7c7bae --fix packages/aws/data_stream/securityhub_findings/**/fields/*yml
[git-generate] elastic-package -C packages/aws build
[git-generate] elastic-package -C packages/aws changelog add --link elastic#14251 --next patch --type bugfix --description 'Modify the data type of `event.kind` from a constant_keyword to a keyword to handle pipeline errors that send `event.kind` to `pipeline_error`.'
f331747 to
948cccf
Compare
[git-generate] elastic-package -C packages/aws changelog add --link elastic#14251 --next patch --type bugfix --description 'Modify the data type of `event.kind` from a constant_keyword to a keyword to handle pipeline errors that send `event.kind` as `pipeline_error`.'
948cccf to
ba5aa86
Compare
|
Pinging @elastic/security-service-integrations (Team:Security-Service Integrations) |
kcreddy
left a comment
There was a problem hiding this comment.
LGTM. but would be nice to get another opinion if the change is non-breaking.
|
From https://www.elastic.co/docs/reference/elasticsearch/mapping-reference/field-data-types
Changing constant_keyword to keyword should not break anything, but the performance characteristics may differ. |
🚀 Benchmarks reportTo see the full report comment with |
💚 Build Succeeded
|
|
? Is there something else that it also closes? |
efd6
left a comment
There was a problem hiding this comment.
LGTM, but probably want to refer to the bug in the PR description.
|
Package aws - 3.8.1 containing this change is available at https://epr.elastic.co/package/aws/3.8.1/ |
Change event.kind to keyword instead of constant_keyword.
When a pipeline error occurs, event.kind was being set to
'pipeline_error' and this break ingestion because event.kind
was already assigned a value of 'state'.
The observed error was:
[constant_keyword] field [event.kind] only accepts values that are equal
to the value defined in the mappings [state], but got [pipeline_error]
Closes elastic#12970
Proposed commit message
Change event.kind to keyword instead of constant_keyword.
When a pipeline error occurs, event.kind was being set to
'pipeline_error' and this break ingestion because event.kind
was already assigned a value of 'state'.
The observed error was:
Closes #12970
Checklist
changelog.ymlfile.Author's Notes
This bug was detected in our internal demo deployment of the integration.
Logs
This will fix errors like:
{ "type": "document_parsing_exception", "reason": "[1:7357] failed to parse field [event.kind] of type [constant_keyword] in document with id '2KqJqX41g3ygdMw5sXCzYx1Wb/Y='. Preview of field's value: 'pipeline_error'", "caused_by": { "type": "illegal_argument_exception", "reason": "[constant_keyword] field [event.kind] only accepts values that are equal to the value defined in the mappings [state], but got [pipeline_error]" } }