Skip to content

Fix durationMs parsing in auditlogs data stream#16674

Merged
Kavindu-Dodan merged 8 commits intoelastic:mainfrom
gregorywychowaniec-zt:main
Dec 29, 2025
Merged

Fix durationMs parsing in auditlogs data stream#16674
Kavindu-Dodan merged 8 commits intoelastic:mainfrom
gregorywychowaniec-zt:main

Conversation

@gregorywychowaniec-zt
Copy link
Contributor

@gregorywychowaniec-zt gregorywychowaniec-zt commented Dec 23, 2025 •
edited
Loading

Proposed commit message

Same as encoutered in #15976, we got some auditlogs with durationMs as a string that break the pipeline. So same fix for auditlogs here :

  • Convert azure.auditlogs.durationMs as long type if the value comes as string

Checklist

  • I have reviewed tips for building integrations and this pull request is aligned with them.
  • I have verified that all data streams collect metrics or logs.
  • I have added an entry to my package's changelog.yml file.
  • I have verified that Kibana version constraints are current according to guidelines.
  • I have verified that any added dashboard complies with Kibana's Dashboard good practices

How to test this PR locally

POST _ingest/pipeline/logs-azure.auditlogs-1.32.1/_simulate
{
"docs": [
{
"_source": {
"message": """
{ "time": "2022-01-22T18:15:02.3875429Z", "resourceId": "/tenants/4bbb79f7-5724-4c9e-95f3-de075f6ec090/providers/Microsoft.aadiam", "operationName": "Update service principal", "operationVersion": "1.0", "category": "AuditLogs", "tenantId": "4bbb79f7-5724-4c9e-95f3-de075f6ec090", "resultSignature": "None", "durationMs": "0", "callerIpAddress": "1.128.3.4", "correlationId": "87979703-118b-498f-99c2-ccd1a56f1a5a", "identity": "Managed Service Identity", "Level": 4, "properties": {"id":"Directory_87979703-118b-498f-99c2-ccd1a56f1a5a_ULAYA_144938566","category":"ApplicationManagement","correlationId":"87979703-118b-498f-99c2-ccd1a56f1a5a","result":"success","resultReason":"","activityDisplayName":"Update service principal","activityDateTime":"2022-01-22T18:15:02.3875429+00:00","loggedByService":"Core Directory","operationType":"Update","userAgent":null,"initiatedBy":{"app":{"appId":null,"displayName":"Managed Service Identity","servicePrincipalId":"b9814691-9ca1-4e55-a1ac-8ef5dd010ec0","servicePrincipalName":null}},"targetResources":[{"id":"a7d5dcbe-0627-4ddf-a2f4-86b6785bcc42","displayName":"billing-test-wus","type":"ServicePrincipal","modifiedProperties":[{"displayName":"TargetId.ServicePrincipalNames","oldValue":null,"newValue":"\"a70a7931-c387-4dce-9f35-fbf95bdcc91e;https://identity.azure.net/N8CUySpCeRFU3iB/PEuFlON4zd8+n8d3qgzrF1MviSY=\""}],"administrativeUnits":[]}],"additionalDetails":[{"key":"User-Agent","value":"Microsoft Azure Graph Client Library 2.1.17-internal"},{"key":"AppId","value":"a70a7931-c387-4dce-9f35-fbf95bdcc91e"}]}}
"""
}
}
]
}

@cla-checker-service
Copy link

cla-checker-service bot commented Dec 23, 2025 •
edited
Loading

💚 CLA has been signed

@zmoog
Copy link
Contributor

zmoog commented Dec 23, 2025

Hey @gregorywychowaniec-zt, thanks for raising this PR! 🙇

I'll open an issue/PR to fix this field across all Azure integrations.

zmoog
zmoog approved these changes Dec 23, 2025
View reviewed changes
Copy link
Contributor

@zmoog zmoog left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, thanks for fixing this @gregorywychowaniec-zt.

@zmoog zmoog added Integration:azure Azure Logs Team:Obs-InfraObs Observability Infrastructure Monitoring team [elastic/obs-infraobs-integrations] Team:Security-Service Integrations Security Service Integrations team [elastic/security-service-integrations] labels Dec 23, 2025
@elasticmachine
Copy link

elasticmachine commented Dec 23, 2025

Pinging @elastic/security-service-integrations (Team:Security-Service Integrations)

@zmoog zmoog added the bugfix Pull request that fixes a bug issue label Dec 23, 2025
@zmoog zmoog self-assigned this Dec 23, 2025
@zmoog
Copy link
Contributor

zmoog commented Dec 23, 2025

I created the issue #16677 to normalize durationMs to long across Azure integrations.

@Kavindu-Dodan
Copy link
Contributor

Kavindu-Dodan commented Dec 23, 2025

@gregorywychowaniec-zt I have resolved the merge conflict and with current approvals, this should be good to go

Kavindu-Dodan and others added 2 commits December 29, 2025 10:20
@Kavindu-Dodan
Merge branch 'main' into main
58cdaa6
@Kavindu-Dodan
fix version and changelog
cc05200 
Signed-off-by: Kavindu Dodanduwa <kavindu.dodanduwa@elastic.co>
@Kavindu-Dodan Kavindu-Dodan enabled auto-merge (squash) December 29, 2025 18:27
@Kavindu-Dodan
fix format
cc73537 
Signed-off-by: Kavindu Dodanduwa <kavindu.dodanduwa@elastic.co>
@Kavindu-Dodan
Copy link
Contributor

Kavindu-Dodan commented Dec 29, 2025

/test

@elastic-vault-github-plugin-prod
Copy link

elastic-vault-github-plugin-prod bot commented Dec 29, 2025

🚀 Benchmarks report

To see the full report comment with /test benchmark fullreport

@Kavindu-Dodan Kavindu-Dodan merged commit dbbf43f into elastic:main Dec 29, 2025
8 checks passed
@elasticmachine
Copy link

elasticmachine commented Dec 29, 2025

💚 Build Succeeded

History

cc @zmoog

@elastic-vault-github-plugin-prod
Copy link

elastic-vault-github-plugin-prod bot commented Dec 29, 2025

Package azure - 1.34.1 containing this change is available at https://epr.elastic.co/package/azure/1.34.1/

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@zmoog zmoog zmoog approved these changes

@Kavindu-Dodan Kavindu-Dodan Kavindu-Dodan approved these changes

@kcreddy kcreddy kcreddy approved these changes

@devamanv devamanv devamanv approved these changes

@ShourieG ShourieG ShourieG approved these changes

Assignees

@zmoog zmoog

Labels

bugfix Pull request that fixes a bug issue Integration:azure Azure Logs Team:Obs-InfraObs Observability Infrastructure Monitoring team [elastic/obs-infraobs-integrations] Team:Security-Service Integrations Security Service Integrations team [elastic/security-service-integrations]

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

7 participants

@gregorywychowaniec-zt @zmoog @elasticmachine @Kavindu-Dodan @kcreddy @devamanv @ShourieG