Allow rerouting of Azure logs events to a different data stream#8432
Merged
zmoog merged 5 commits intoelastic:mainfrom Nov 8, 2023
Merged
Allow rerouting of Azure logs events to a different data stream#8432zmoog merged 5 commits intoelastic:mainfrom
zmoog merged 5 commits intoelastic:mainfrom
Conversation
This will allow us to: - Send all the log events to a single data stream (for example, the generic event hub, or platform logs) and then route them to another one. - Remove the embedded Spring Apps pipeline from platform logs.
The reroute processor [^1] is available starting from stack version 8.8.0. [^1]: https://www.elastic.co/guide/en/elasticsearch/reference/master/reroute-processor.html
💚 Build Succeeded
Expand to view the summary
Build stats
Test stats 🧪
🤖 GitHub commentsExpand to view the GitHub comments
To re-run your PR in the CI, just comment with:
|
🌐 Coverage report
|
kaiyan-sheng
approved these changes
Nov 8, 2023
Contributor
Author
|
The test fails because one of the platform logs test files contains invalid JSON. The test wasn't failing before because up to stack version 8.6, the JSON processor was tolerant of invalid documents. From 8.7, the JSON became strict by default, and now that I bumped the minimum version 8.8, it fails. For all the gory details, check out zmoog/public-notes#54 |
With 8.7, the JSON processor became strict by default; this caused the failure of the platform logs due to an old, invalid JSON document lurking around in the test suite.
|
Package azure - 1.8.0 containing this change is available at https://epr.elastic.co/search?package=azure |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
4 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Proposed commit message
Add permissions to reroute log events in all data streams in Azure Logs.
The additional permissions will allow us to:
Rerouting can be set up by defining routing rules at the integration level or allowing users to add a reroute processor to the custom ingest pipeline.
Checklist
changelog.ymlfile.