Skip to content

Add support for sprincloud logs inside the platformcloudlogs pipeline #1462

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
narph merged 9 commits into from
Aug 11, 2021
Merged

Add support for sprincloud logs inside the platformcloudlogs pipeline #1462

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
9 commits
Select commit Hold shift + click to select a range
13a735b
add support for sprincloud logs
narph Aug 9, 2021
9e3c5db
pipeline
narph Aug 10, 2021
a4b2571
formatting
narph Aug 10, 2021
ef0737f
fix
narph Aug 10, 2021
a3f5ae2
update pipeline
narph Aug 10, 2021
5f08abb
add logs
narph Aug 10, 2021
34c33f2
test
narph Aug 10, 2021
1e708f8
test
narph Aug 10, 2021
0bf6337
remove tests
narph Aug 10, 2021
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions packages/azure/changelog.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,4 +1,9 @@
# newer versions go on top
- version: "0.8.1"
changes:
- description: Add support for springcloud logs inside the platformlogs pipeline
type: enhancement
link: https://github.com/elastic/integrations/pull/1462
- version: "0.8.0"
changes:
- description: Import ECS field definitions
Expand Down
2 changes: 1 addition & 1 deletion ...es/azure/data_stream/platformlogs/elasticsearch/ingest_pipeline/azure-shared-pipeline.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
---
description: Pipeline for parsing azure activity logs.
description: Pipeline for parsing azure logs.
processors:
- set:
field: cloud.provider
Expand Down
3 changes: 3 additions & 0 deletions packages/azure/data_stream/platformlogs/elasticsearch/ingest_pipeline/default.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -245,6 +245,9 @@ processors:
if: "ctx?.tags == null || !(ctx.tags.contains('preserve_original_event'))"
ignore_failure: true
ignore_missing: true
- pipeline:
if: "ctx?.azure_log_forwarder?.resource_type != null && (ctx?.azure_log_forwarder?.resource_type == 'Microsoft.AppPlatform/Spring'|| ctx?.azure_log_forwarder?.resource_type == 'MICROSOFT.APPPLATFORM/SPRING')"
name: '{{ IngestPipeline "springcloudlogs-inner-pipeline" }}'
on_failure:
- set:
field: error.message
Expand Down
86 changes: 86 additions & 0 deletions ...data_stream/platformlogs/elasticsearch/ingest_pipeline/springcloudlogs-inner-pipeline.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,86 @@
---
description: Pipeline for parsing azure spring cloud logs.
processors:
- script:
source: "ctx['_index'] = ctx['_index'].replace('platformlogs', 'springcloudlogs')"
ignore_failure: true
- rename:
field: azure.platformlogs
target_field: azure.springcloudlogs
ignore_missing: true
- set:
field: event.dataset
value: azure.springcloudlogs
ignore_failure: true
- set:
field: data_stream.dataset
value: azure.springcloudlogs
ignore_failure: true
- drop:
if: "ctx.azure.springcloudlogs.category != 'SystemLogs' && ctx.azure.springcloudlogs.category != 'ApplicationConsole'"
ignore_failure: true
- rename:
field: azure.springcloudlogs.LogFormat
target_field: azure.springcloudlogs.log_format
ignore_missing: true
- rename:
field: azure.springcloudlogs.properties.InstanceName
target_field: azure.springcloudlogs.properties.instance_name
ignore_missing: true
- rename:
field: azure.springcloudlogs.properties.Log
target_field: azure.springcloudlogs.properties.log
ignore_missing: true
- rename:
field: azure.springcloudlogs.properties.ServiceName
target_field: azure.springcloudlogs.properties.service_name
ignore_missing: true
- rename:
field: azure.springcloudlogs.properties.Stream
target_field: azure.springcloudlogs.properties.stream
ignore_missing: true
- rename:
field: azure.springcloudlogs.properties.AppName
target_field: azure.springcloudlogs.properties.app_name
ignore_missing: true
- rename:
field: azure.springcloudlogs.properties.ServiceId
target_field: azure.springcloudlogs.properties.service_id
ignore_missing: true
- rename:
field: azure.springcloudlogs.properties.Type
target_field: azure.springcloudlogs.properties.type
ignore_missing: true
- rename:
field: azure.springcloudlogs.properties.Level
target_field: azure.springcloudlogs.level
ignore_missing: true
- rename:
field: azure.springcloudlogs.properties.Logger
target_field: azure.springcloudlogs.properties.logger
ignore_missing: true
- rename:
field: azure.springcloudlogs.properties.Stack
target_field: azure.springcloudlogs.properties.stack
ignore_missing: true
- rename:
field: azure.springcloudlogs.properties.Thread
target_field: azure.springcloudlogs.properties.thread
ignore_missing: true
- rename:
field: azure.springcloudlogs.level
target_field: log.level
ignore_missing: true
- rename:
field: azure.springcloudlogs.operationName
target_field: azure.springcloudlogs.operation_name
ignore_missing: true
- convert:
field: azure.springcloudlogs.operation_name
target_field: event.action
type: string
ignore_missing: true
on_failure:
- set:
field: error.message
value: '{{ _ingest.on_failure_message }}'
2 changes: 1 addition & 1 deletion packages/azure/manifest.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
name: azure
title: Azure Logs
version: 0.8.0
version: 0.8.1
release: beta
description: This Elastic integration collects logs from Azure
type: integration
Expand Down