Skip to content

[Azure Logs] Fix authentication_processing_details parsing in sign-in logs#5129

Merged
zmoog merged 3 commits intoelastic:mainfrom
zmoog:zmoog/azure-authentication-processing-details-issue
Feb 7, 2023
Merged

[Azure Logs] Fix authentication_processing_details parsing in sign-in logs#5129
zmoog merged 3 commits intoelastic:mainfrom
zmoog:zmoog/azure-authentication-processing-details-issue

Conversation

@zmoog
Copy link
Contributor

@zmoog zmoog commented Jan 27, 2023

What does this PR do?

What

Replace the foreach + set processors combo with a single script processor in the ingest pipeline for Azure AD sign-in logs.

Why

The set processor expands the dots contained in the field name into subfields.

Sometimes attributes contained in authentication_processing_details have dots, for example:

# source
{"key": "a.b.c", "value": true}

In such cases, the set processor would turn it into:

# this is a side-effect
{
  "a": {
    "b": {
      "c": true
    }
  }
}

Instead of:

# this is the expected result
{"a.b.c": True}

Checklist

  • I have reviewed tips for building integrations and this pull request is aligned with them.
  • I have verified that all data streams collect metrics or logs.
  • I have added an entry to my package's changelog.yml file.
  • I have verified that Kibana version constraints are current according to guidelines.

Author's Checklist

  • [ ]

How to test this PR locally

Related issues

Screenshots

@zmoog zmoog self-assigned this Jan 27, 2023
@zmoog zmoog added bug Something isn't working, use only for issues Team:Cloud-Monitoring Label for the Cloud Monitoring team labels Jan 27, 2023
@zmoog zmoog changed the title Fix authentication_processing_details parsing [Azure Logs] Fix authentication_processing_details parsing Jan 27, 2023
@zmoog zmoog changed the title [Azure Logs] Fix authentication_processing_details parsing [Azure Logs] Fix authentication_processing_details parsing in sign-in logs Jan 27, 2023
@elasticmachine
Copy link

elasticmachine commented Jan 27, 2023

💚 Build Succeeded

the below badges are clickable and redirect to their specific view in the CI or DOCS
Pipeline View Test View Changes Artifacts preview preview

Expand to view the summary

Build stats

  • Start Time: 2023-02-07T13:56:57.456+0000

  • Duration: 15 min 11 sec

Test stats 🧪

Test Results
Failed 0
Passed 124
Skipped 0
Total 124

🤖 GitHub comments

Expand to view the GitHub comments

To re-run your PR in the CI, just comment with:

  • /test : Re-trigger the build.

@elasticmachine
Copy link

elasticmachine commented Jan 27, 2023

🌐 Coverage report

Name Metrics % (covered/total) Diff
Packages 100.0% (10/10) 💚
Files 86.364% (19/22) 👎 -13.636
Classes 86.364% (19/22) 👎 -13.636
Methods 83.333% (155/186) 👎 -16.667
Lines 85.047% (2787/3277) 👎 -7.26
Conditionals 100.0% (0/0) 💚

@UcanInfosec
Copy link

If ready, let’s move forward with this

@zmoog zmoog marked this pull request as ready for review February 3, 2023 10:54
@zmoog zmoog requested a review from a team as a code owner February 3, 2023 10:54
zmoog and others added 3 commits February 7, 2023 14:52
Turns the `authentication_processing_details` array elements into
key/value pairs without creating sub-fields if the key contains one
or more dots.

For example, the array element:

  `{"key": "TLS 1.0", "value": False}`

Becomes:

  `{"TLS 1.0": "False"}`

Previously the `set` processor would have created an unintended sub
field:

  `{"TLS 1": {".0": "False"}}`

I know it's weird having dots in the keys, but it's the way it is.
…eline/default.yml


Thank you @andrewkroh!

Co-authored-by: Andrew Kroh <andrew.kroh@elastic.co>
@zmoog zmoog force-pushed the zmoog/azure-authentication-processing-details-issue branch from cd68425 to aeaf106 Compare February 7, 2023 13:56
@zmoog zmoog merged commit d84c658 into elastic:main Feb 7, 2023
@zmoog zmoog deleted the zmoog/azure-authentication-processing-details-issue branch February 7, 2023 14:42
@elasticmachine
Copy link

Package azure - 1.5.7 containing this change is available at https://epr.elastic.co/search?package=azure

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working, use only for issues Team:Cloud-Monitoring Label for the Cloud Monitoring team

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants