Skip to content

[sysdig] Add support for cspm datastream#14907

Merged
brijesh-elastic merged 10 commits intoelastic:mainfrom
brijesh-elastic:sysdig-cspm
Sep 18, 2025
Merged

[sysdig] Add support for cspm datastream#14907
brijesh-elastic merged 10 commits intoelastic:mainfrom
brijesh-elastic:sysdig-cspm

Conversation

@brijesh-elastic
Copy link
Collaborator

@brijesh-elastic brijesh-elastic commented Aug 12, 2025

Proposed commit message

sysdig: add support for cspm data stream to collect compliance results.

The CSPM data stream logs provide an overview of the evaluation results of your
Cloud and Kubernetes environment’s adherence to specific security standards,
regulations, and policies. The findings highlight areas where your organization
is meeting or failing to meet the required security controls and procedures.

Sanitized test case inputs were obtained from live Sysdig Secure instance
using the Sysdig API.

Note

The dashboard will be added once the vulnerability PR is merged.

Checklist

  • I have reviewed tips for building integrations and this pull request is aligned with them.
  • I have verified that all data streams collect metrics or logs.
  • I have added an entry to my package's changelog.yml file.
  • I have verified that Kibana version constraints are current according to guidelines.
  • I have verified that any added dashboard complies with Kibana's Dashboard good practices

How to test this PR locally

  • Clone integrations repo.
  • Install elastic package locally.
  • Start elastic stack using elastic-package.
  • Move to integrations/packages/sysdig directory.
  • Run the following command to run tests.

elastic-package test

Related issues

@brijesh-elastic brijesh-elastic self-assigned this Aug 12, 2025
@brijesh-elastic brijesh-elastic requested a review from a team as a code owner August 12, 2025 10:14
@brijesh-elastic brijesh-elastic added documentation Improvements or additions to documentation. Applied to PRs that modify *.md files. enhancement New feature or request Team:Security-Service Integrations Security Service Integrations team [elastic/security-service-integrations] Integration:sysdig Sysdig Team:Sit-Crest Crest developers on the Security Integrations team [elastic/sit-crest-contractors] labels Aug 12, 2025
@elasticmachine
Copy link

elasticmachine commented Aug 12, 2025

Pinging @elastic/security-service-integrations (Team:Security-Service Integrations)

@elastic-vault-github-plugin-prod
Copy link

elastic-vault-github-plugin-prod bot commented Aug 12, 2025 •
edited
Loading

🚀 Benchmarks report

To see the full report comment with /test benchmark fullreport

efd6
efd6 reviewed Aug 14, 2025
View reviewed changes
Copy link
Contributor

@efd6 efd6 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit only, otherwise LGTM

efd6
efd6 previously approved these changes Sep 7, 2025
View reviewed changes
@brijesh-elastic brijesh-elastic marked this pull request as draft September 8, 2025 05:50
@brijesh-elastic brijesh-elastic marked this pull request as ready for review September 13, 2025 10:05
efd6
efd6 reviewed Sep 14, 2025
View reviewed changes
@efd6 efd6 dismissed their stale review September 14, 2025 23:39

out of date

@andrewkroh andrewkroh added the dashboard Relates to a Kibana dashboard bug, enhancement, or modification. label Sep 15, 2025
kcreddy
kcreddy reviewed Sep 15, 2025
View reviewed changes
packages/sysdig/data_stream/cspm/_dev/test/pipeline/test-cspm.log-expected.json
@@ -0,0 +1,702 @@
{
Copy link
Contributor

@kcreddy kcreddy Sep 15, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you try best to all Must Have mappings from https://docs.elastic.dev/security-solution/cloud-security/cdr/3p-dev-guide#misconfiguration-findings-1 ?

Copy link
Collaborator Author

@brijesh-elastic brijesh-elastic Sep 16, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Most of the MUST HAVE fields (such as resource.id, resource.name, host.name, user.name) aren't possible because resource details aren't coming in the API response.

Copy link
Contributor

@kcreddy kcreddy Sep 16, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It seems we are only ingesting a bunch of rules. Is there any other API to ingest resources from that are evaluated by these rules?
I don't think ingesting rules without resources will add any value.

Copy link
Collaborator Author

@brijesh-elastic brijesh-elastic Sep 16, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We are receiving the resource_api_endpoint field, which can be used to call another API to get the affected resources. However, since we have 40k controls, this would result in 40k calls in one iteration, which is excessively high.

Copy link
Contributor

@kcreddy kcreddy Sep 16, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There is summary information inside *Count fields, indicating overall pass/fail along with severities.

I will leave it to @chemamartinez if the requirement is satisfied for this data stream.

Copy link
Contributor

@chemamartinez chemamartinez Sep 17, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added a comment at the issue.

kcreddy
kcreddy reviewed Sep 15, 2025
View reviewed changes
efd6
efd6 reviewed Sep 16, 2025
View reviewed changes
Copy link
Contributor

@efd6 efd6 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM after @kcreddy's concerns are addressed.

@elasticmachine
Copy link

elasticmachine commented Sep 16, 2025

💚 Build Succeeded

History

cc @brijesh-elastic

@elastic-sonarqube
Copy link

elastic-sonarqube bot commented Sep 16, 2025

Quality Gate passed Quality Gate passed

Issues
0 New issues
0 Fixed issues
0 Accepted issues

Measures
0 Security Hotspots
97.8% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarQube

kcreddy
kcreddy reviewed Sep 16, 2025
View reviewed changes
Copy link
Contributor

@kcreddy kcreddy left a comment •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Waiting on #14907 (comment). Rest LGTM.

@kcreddy kcreddy mentioned this pull request Sep 16, 2025
Closed
kcreddy
kcreddy approved these changes Sep 17, 2025
View reviewed changes
Copy link
Contributor

@kcreddy kcreddy left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@brijesh-elastic brijesh-elastic merged commit 03b6ee7 into elastic:main Sep 18, 2025
9 checks passed
@elastic-vault-github-plugin-prod
Copy link

elastic-vault-github-plugin-prod bot commented Sep 18, 2025

Package sysdig - 2.2.0 containing this change is available at https://epr.elastic.co/package/sysdig/2.2.0/

@brijesh-elastic brijesh-elastic mentioned this pull request Oct 8, 2025
Open
35 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@chemamartinez chemamartinez chemamartinez left review comments

@efd6 efd6 efd6 left review comments

@kcreddy kcreddy kcreddy approved these changes

Assignees

@brijesh-elastic brijesh-elastic

Labels

dashboard Relates to a Kibana dashboard bug, enhancement, or modification. documentation Improvements or additions to documentation. Applied to PRs that modify *.md files. enhancement New feature or request Integration:sysdig Sysdig Team:Security-Service Integrations Security Service Integrations team [elastic/security-service-integrations] Team:Sit-Crest Crest developers on the Security Integrations team [elastic/sit-crest-contractors]

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

[Sysdig Secure] New data stream: CSPM

6 participants

@brijesh-elastic @elasticmachine @kcreddy @chemamartinez @efd6 @andrewkroh