feat(topology): SNMP LLDP/CDP and NetFlow/IPFIX/sFlow v5 collectors with topology functions#21702
feat(topology): SNMP LLDP/CDP and NetFlow/IPFIX/sFlow v5 collectors with topology functions#21702ktsaou wants to merge 64 commits intonetdata:masterfrom
Conversation
There was a problem hiding this comment.
3 issues found across 218 files
Confidence score: 3/5
- The maxBuckets cap in
src/go/plugin/go.d/collector/netflow/flow_aggregator.gocan be bypassed for older buckets, risking unbounded map growth and memory pressure. src/go/plugin/go.d/collector/netflow/collector.goignores caller contexts in Init/Cleanup, so framework cancellation/timeout may not propagate, impacting shutdown reliability.- Score reflects concrete behavioral risks (resource growth and shutdown propagation), though fixes are localized.
- Pay close attention to
src/go/plugin/go.d/collector/netflow/flow_aggregator.go,src/go/plugin/go.d/collector/netflow/collector.go- bucket limit enforcement and context propagation.
Note: This PR contains a large number of files. cubic only reviews up to 75 files per PR, so some files may not have been reviewed.
Prompt for AI agents (all issues)
Check if these issues are valid — if so, understand the root cause of each and fix them.
<file name="src/go/plugin/go.d/collector/netflow/collector.go">
<violation number="1" location="src/go/plugin/go.d/collector/netflow/collector.go:117">
P2: Init discards the caller context and always uses context.Background(), so module cancellation from the framework won't propagate to the collector goroutines.</violation>
<violation number="2" location="src/go/plugin/go.d/collector/netflow/collector.go:158">
P2: Cleanup ignores the caller’s context and uses context.Background(), which can block shutdowns when the framework expects cancellation/timeout to be honored.</violation>
</file>
<file name="src/go/plugin/go.d/collector/netflow/flow_aggregator.go">
<violation number="1" location="src/go/plugin/go.d/collector/netflow/flow_aggregator.go:255">
P2: maxBuckets enforcement can fail when the incoming bucket is older than all existing buckets, allowing the map to exceed the configured limit.</violation>
</file>
Reply with feedback, questions, or to request a fix. Tag @cubic-dev-ai to re-run a review.
There was a problem hiding this comment.
3 issues found across 128 files (changes from recent commits).
Note: This PR contains a large number of files. cubic only reviews up to 75 files per PR, so some files may not have been reviewed.
Prompt for AI agents (all issues)
Check if these issues are valid — if so, understand the root cause of each and fix them.
<file name="src/crates/netdata-netflow/netflow-plugin/src/network_sources.rs">
<violation number="1" location="src/crates/netdata-netflow/netflow-plugin/src/network_sources.rs:178">
P2: Silent failure on lock poisoning: if `state.by_source.write()` fails due to lock poisoning, records are silently dropped without logging. Consider at minimum logging a warning when the lock cannot be acquired.</violation>
</file>
<file name="src/crates/netdata-netflow/netflow-plugin/src/plugin_config.rs">
<violation number="1" location="src/crates/netdata-netflow/netflow-plugin/src/plugin_config.rs:984">
P2: The gRPC address validation only checks for any ':' when no scheme is provided, so IPv6 literals without a port pass validation even though the error message requires host:port. This lets invalid configs through and will fail later at connection time.</violation>
</file>
<file name="src/go/plugin/go.d/collector/snmp/topology_cache_test.go">
<violation number="1" location="src/go/plugin/go.d/collector/snmp/topology_cache_test.go:203">
P3: The helper treats any non-nil attribute type as a valid non-empty list, so the test can pass even when the attribute is the wrong type or empty. To keep the test meaningful, only return true for non-empty list types and treat unknown types as missing.</violation>
</file>
Reply with feedback, questions, or to request a fix. Tag @cubic-dev-ai to re-run a review.
src/crates/netdata-netflow/netflow-plugin/src/network_sources.rs
Outdated
Show resolved
Hide resolved
There was a problem hiding this comment.
1 issue found across 4 files (changes from recent commits).
Prompt for AI agents (all issues)
Check if these issues are valid — if so, understand the root cause of each and fix them.
<file name="src/web/api/functions/function-streaming.c">
<violation number="1" location="src/web/api/functions/function-streaming.c:18">
P1: Avoid unbounded variable-length stack allocations from the user-controlled `function` string; a large request can exhaust the stack and crash the agent. Use a bounded buffer with a length check or heap allocation with proper cleanup.</violation>
</file>
Reply with feedback, questions, or to request a fix. Tag @cubic-dev-ai to re-run a review.
ee1bb5a to
70cb670
Compare
src/crates/netdata-netflow/netflow-plugin/src/network_sources.rs
Dismissed
Show dismissed
Hide dismissed
There was a problem hiding this comment.
Pull request overview
This PR adds comprehensive Layer 2 topology discovery via SNMP (LLDP/CDP) and Layer 3 flow collection via NetFlow/IPFIX/sFlow v5, along with corresponding topology functions for dashboard integration. It also includes a CLI aggregation tool and extensive test coverage.
Changes:
- SNMP topology: LLDP-MIB and CISCO-CDP-MIB neighbor discovery with cached topology and
topology:snmpfunction - NetFlow plugin (Rust): UDP listeners, decoders (v5/v7/v9, IPFIX, sFlow v5), tiered journal storage, and
flows:netflowfunction - Function API: Added
ResponseTypefield to support non-table schemas (topology, flows) - Journal writer: Monotonic timestamp clamping with restart seeding for safe timestamp overrides
- Network Viewer: Added
topology:network-viewerfunction for live L7 topology
Reviewed changes
Copilot reviewed 61 out of 237 changed files in this pull request and generated 7 comments.
Show a summary per file
| File | Description |
|---|---|
| src/go/plugin/go.d/collector/snmp/testdata/config.yaml | Added topology autoprobe configuration |
| src/go/plugin/go.d/collector/snmp/testdata/ATTRIBUTION.md | Attribution for 116 LibreNMS SNMP fixtures |
| src/go/plugin/go.d/collector/snmp/metadata.yaml | Added topology function metadata and autoprobe config |
| src/go/plugin/go.d/collector/snmp/func_topology.go | New topology function handler for SNMP |
| src/go/plugin/go.d/collector/snmp/func_router.go | Added topology handler registration |
| src/go/plugin/go.d/collector/snmp/ddsnmp/profile.go | Added FinalizeProfiles and HasExtension helpers |
| src/go/plugin/go.d/collector/snmp/config.go | Added TopologyConfig struct |
| src/go/plugin/go.d/agent/jobmgr/funcshandler.go | Added ResponseType support in function responses |
| src/go/pkg/topology/types.go | New unified topology schema types |
| src/go/pkg/funcapi/response.go | Added ResponseType field to MethodConfig and FunctionResponse |
| src/crates/netdata-netflow/netflow-plugin/src/tiering.rs | New tiered flow aggregation (1m/5m/1h) |
| src/crates/netdata-netflow/netflow-plugin/src/routing_bioris.rs | BioRIS gRPC routing enrichment client |
| src/crates/netdata-netflow/netflow-plugin/src/rollup.rs | Rollup key generation for aggregated tiers |
| src/crates/netdata-netflow/netflow-plugin/src/network_sources.rs | Remote network source fetcher with jq transforms |
| src/crates/netdata-netflow/netflow-plugin/src/main.rs | NetFlow plugin binary with flows function handler |
| src/crates/netdata-netflow/netflow-plugin/src/ingest.rs | Flow ingestion service with journal writers |
| src/crates/netdata-netflow/netflow-plugin/proto/*.proto | BioRIS protobuf definitions |
| src/crates/netdata-netflow/netflow-plugin/configs/netflow.yaml | Default NetFlow plugin configuration |
| src/crates/netdata-netflow/netflow-plugin/build.rs | Protobuf compilation build script |
| src/crates/netdata-netflow/netflow-plugin/README.md | NetFlow plugin documentation |
| src/crates/journal-log-writer/src/log/mod.rs | Added EntryTimestamps with monotonic clamping |
| src/crates/journal-log-writer/src/log/config.rs | Added machine_id_suffix configuration |
| src/crates/journal-log-writer/src/log/chain.rs | Added tail_monotonic_for_boot for restart seeding |
| src/crates/journal-engine/src/logs/query.rs | Added output_fields projection for query optimization |
| src/crates/journal-common/src/time.rs | Added RealtimeClock::observe for monotonic timestamp observation |
| src/collectors/network-viewer.plugin/network-viewer.c | Added topology:network-viewer function for L7 topology |
| packaging/makeself/install-or-update.sh | Added netflow-plugin to installer permissions |
| packaging/installer/functions.sh | Added ENABLE_PLUGIN_NETFLOW CMake option |
| packaging/cmake/pkg-files/deb/plugin-netflow/postinst | Debian postinst for netflow-plugin permissions |
| packaging/cmake/Modules/Packaging.cmake | Added netflow-plugin component packaging |
| CMakeLists.txt | Added netflow-plugin build integration |
| .github/workflows/snmp-netflow-sim-tests.yml | CI workflow for SNMP simulator tests |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
src/crates/netdata-netflow/netflow-plugin/src/routing_bioris.rs
Outdated
Show resolved
Hide resolved
There was a problem hiding this comment.
2 issues found across 2 files (changes from recent commits).
Prompt for AI agents (all issues)
Check if these issues are valid — if so, understand the root cause of each and fix them.
<file name="src/go/plugin/go.d/collector/snmp/integrations/snmp_devices.md">
<violation number="1" location="src/go/plugin/go.d/collector/snmp/integrations/snmp_devices.md:208">
P2: The security description for `Snmp:topology` incorrectly claims it only exposes interface names and traffic counters, but this function returns LLDP/CDP topology data (devices, links, management addresses). This is misleading for users reading the documentation.</violation>
<violation number="2" location="src/go/plugin/go.d/collector/snmp/integrations/snmp_devices.md:209">
P2: The availability description references interface data caching instead of LLDP/CDP topology data, which doesn’t match this function’s purpose. It should reference topology cache readiness.</violation>
</file>
Reply with feedback, questions, or to request a fix. Tag @cubic-dev-ai to re-run a review.
There was a problem hiding this comment.
3 issues found across 10 files (changes from recent commits).
Prompt for AI agents (all issues)
Check if these issues are valid — if so, understand the root cause of each and fix them. If appropriate, use sub-agents to investigate and fix each issue separately.
<file name="src/crates/journalctl/src/main.rs">
<violation number="1" location="src/crates/journalctl/src/main.rs:152">
P2: Guard against relative times that underflow the UNIX epoch; the current subtraction can go negative and wrap when cast to u64.</violation>
</file>
<file name="src/crates/journal-session/src/cursor.rs">
<violation number="1" location="src/crates/journal-session/src/cursor.rs:103">
P2: Broken rustdoc intra-doc link: `Cursor::fields` does not exist. The method is `payloads()`, so the link target should be `Cursor::payloads`.</violation>
<violation number="2" location="src/crates/journal-session/src/cursor.rs:475">
P2: This `expect()` panics in library code, which violates the project convention ("Avoid panics in library paths; return typed errors instead"). Additionally, the panic message references `fields()` but the method is named `payloads()`. Return a typed `SessionError` variant instead.</violation>
</file>
Reply with feedback, questions, or to request a fix. Tag @cubic-dev-ai to re-run a review.
| }; | ||
|
|
||
| let now = chrono::Utc::now().timestamp_micros(); | ||
| return Ok((now - offset_usec) as u64); |
There was a problem hiding this comment.
P2: Guard against relative times that underflow the UNIX epoch; the current subtraction can go negative and wrap when cast to u64.
Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. At src/crates/journalctl/src/main.rs, line 152:
<comment>Guard against relative times that underflow the UNIX epoch; the current subtraction can go negative and wrap when cast to u64.</comment>
<file context>
@@ -0,0 +1,432 @@
+ };
+
+ let now = chrono::Utc::now().timestamp_micros();
+ return Ok((now - offset_usec) as u64);
+ }
+
</file context>
| /// | ||
| /// Must be called after [`step()`](Cursor::step) returns `true`. | ||
| pub fn payloads(&mut self) -> Result<Payloads<'_>, SessionError> { | ||
| let entry_offset = self.entry_offset.expect("fields() called without step()"); |
There was a problem hiding this comment.
P2: This expect() panics in library code, which violates the project convention ("Avoid panics in library paths; return typed errors instead"). Additionally, the panic message references fields() but the method is named payloads(). Return a typed SessionError variant instead.
Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. At src/crates/journal-session/src/cursor.rs, line 475:
<comment>This `expect()` panics in library code, which violates the project convention ("Avoid panics in library paths; return typed errors instead"). Additionally, the panic message references `fields()` but the method is named `payloads()`. Return a typed `SessionError` variant instead.</comment>
<file context>
@@ -0,0 +1,517 @@
+ ///
+ /// Must be called after [`step()`](Cursor::step) returns `true`.
+ pub fn payloads(&mut self) -> Result<Payloads<'_>, SessionError> {
+ let entry_offset = self.entry_offset.expect("fields() called without step()");
+ let jf = self.journal_file.as_ref().unwrap();
+ let iter = jf.entry_data_objects(entry_offset)?;
</file context>
301482a to
e033503
Compare
There was a problem hiding this comment.
2 issues found across 9 files (changes from recent commits).
Prompt for AI agents (all issues)
Check if these issues are valid — if so, understand the root cause of each and fix them. If appropriate, use sub-agents to investigate and fix each issue separately.
<file name="src/go/plugin/go.d/agent/jobmgr/manager.go">
<violation number="1" location="src/go/plugin/go.d/agent/jobmgr/manager.go:174">
P2: The new topology alias registration isn't mirrored in cleanup; `topology:*` aliases remain registered after shutdown, leaving stale handlers if the manager is re-initialized. Add corresponding unregistration for the alias names.</violation>
</file>
<file name="src/go/plugin/go.d/collector/snmp/ddsnmp/ddsnmpcollector/collector_table.go">
<violation number="1" location="src/go/plugin/go.d/collector/snmp/ddsnmp/ddsnmpcollector/collector_table.go:330">
P1: Caching auxiliary tables (tables with no symbols, used only for cross-table tags) prevents them from being re-walked. However, dependent tables that *do* need walking (e.g., due to cache expiry or first run) rely on `walkedData` to resolve cross-table tags. Since `walkedData` only contains the results of the *current* walk, skipping the auxiliary table means its data is missing from `walkedData`, causing tag resolution to fail for the dependent table.
The fix is to ensure auxiliary tables are always present in `walkedData` when needed. The safest way with the current architecture is to NOT cache them (remove the `cacheData` call), forcing them to be walked every cycle. Alternatively, the cross-table resolution logic would need to be updated to fallback to reading from the cache.</violation>
</file>
Reply with feedback, questions, or to request a fix. Tag @cubic-dev-ai to re-run a review.
src/go/plugin/go.d/collector/snmp/ddsnmp/ddsnmpcollector/collector_table.go
Show resolved
Hide resolved
| // Register default module-scoped function name. | ||
| funcNames := []string{fmt.Sprintf("%s:%s", name, method.ID)} | ||
| // Topology methods may also be exposed with topology:* aliases for UI grouping. | ||
| if method.ResponseType == "topology" && strings.HasPrefix(method.ID, "topology:") { |
There was a problem hiding this comment.
P2: The new topology alias registration isn't mirrored in cleanup; topology:* aliases remain registered after shutdown, leaving stale handlers if the manager is re-initialized. Add corresponding unregistration for the alias names.
Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. At src/go/plugin/go.d/agent/jobmgr/manager.go, line 174:
<comment>The new topology alias registration isn't mirrored in cleanup; `topology:*` aliases remain registered after shutdown, leaving stale handlers if the manager is re-initialized. Add corresponding unregistration for the alias names.</comment>
<file context>
@@ -168,10 +168,13 @@ func (m *Manager) Run(ctx context.Context, in chan []*confgroup.Group) {
+ // Register default module-scoped function name.
+ funcNames := []string{fmt.Sprintf("%s:%s", name, method.ID)}
+ // Topology methods may also be exposed with topology:* aliases for UI grouping.
+ if method.ResponseType == "topology" && strings.HasPrefix(method.ID, "topology:") {
+ funcNames = append(funcNames, method.ID)
+ }
</file context>
There was a problem hiding this comment.
1 issue found across 1 file (changes from recent commits).
Prompt for AI agents (all issues)
Check if these issues are valid — if so, understand the root cause of each and fix them. If appropriate, use sub-agents to investigate and fix each issue separately.
<file name="src/go/tools/topology-parity-evidence/main.go">
<violation number="1">
P2: Assertion detection uses `rawLine`, so `assert*` tokens in comments or string literals are counted as real assertions, which can inflate the inventory and create false parity gaps. Scan `cleanLine` (the comment/string-stripped version) instead.</violation>
</file>
Reply with feedback, questions, or to request a fix. Tag @cubic-dev-ai to re-run a review.
| @@ -0,0 +1,2806 @@ | |||
| // SPDX-License-Identifier: GPL-3.0-or-later | |||
There was a problem hiding this comment.
P2: Assertion detection uses rawLine, so assert* tokens in comments or string literals are counted as real assertions, which can inflate the inventory and create false parity gaps. Scan cleanLine (the comment/string-stripped version) instead.
Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. At src/go/tools/topology-parity-evidence/main.go, line 2286:
<comment>Assertion detection uses `rawLine`, so `assert*` tokens in comments or string literals are counted as real assertions, which can inflate the inventory and create false parity gaps. Scan `cleanLine` (the comment/string-stripped version) instead.</comment>
<file context>
@@ -0,0 +1,2806 @@
+ inBlockComment = nextInBlockComment
+ trimmed := strings.TrimSpace(cleanLine)
+
+ if matches := assertionCallRE.FindAllStringSubmatchIndex(rawLine, -1); len(matches) > 0 {
+ for _, m := range matches {
+ if len(m) < 4 {
</file context>
There was a problem hiding this comment.
3 issues found across 25 files (changes from recent commits).
Prompt for AI agents (all issues)
Check if these issues are valid — if so, understand the root cause of each and fix them. If appropriate, use sub-agents to investigate and fix each issue separately.
<file name="src/go/pkg/topology/engine/parity/evidence/office-live-reliability-report.md">
<violation number="1" location="src/go/pkg/topology/engine/parity/evidence/office-live-reliability-report.md:6">
P2: Avoid committing real SNMP community strings in documentation; replace with a redacted placeholder to prevent leaking credentials.</violation>
</file>
<file name="src/go/plugin/go.d/collector/snmp/topology_registry.go">
<violation number="1" location="src/go/plugin/go.d/collector/snmp/topology_registry.go:177">
P2: L3 topology responses skip augmentLocalActorFromCache, so local device attributes (management addresses, capabilities, sys_descr, etc.) are missing compared with L2. Consider augmenting the L3 data before returning.</violation>
<violation number="2" location="src/go/plugin/go.d/collector/snmp/topology_registry.go:182">
P2: Merged topology responses skip augmentLocalActorFromCache, so local device attributes are dropped in the combined L2/L3 output. Consider augmenting the merged data before returning.</violation>
</file>
Reply with feedback, questions, or to request a fix. Tag @cubic-dev-ai to re-run a review.
| ## Scope | ||
| - Gate: `T4.5` + `T4.6` | ||
| - Office seed: `10.20.4.0/24` | ||
| - Community: `atadteN` |
There was a problem hiding this comment.
P2: Avoid committing real SNMP community strings in documentation; replace with a redacted placeholder to prevent leaking credentials.
Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. At src/go/pkg/topology/engine/parity/evidence/office-live-reliability-report.md, line 6:
<comment>Avoid committing real SNMP community strings in documentation; replace with a redacted placeholder to prevent leaking credentials.</comment>
<file context>
@@ -0,0 +1,60 @@
+## Scope
+- Gate: `T4.5` + `T4.6`
+- Office seed: `10.20.4.0/24`
+- Community: `atadteN`
+- Function under validation: `topology:snmp` (`topology_view:l2|l3|merged`)
+
</file context>
There was a problem hiding this comment.
1 issue found across 7 files (changes from recent commits).
Prompt for AI agents (all issues)
Check if these issues are valid — if so, understand the root cause of each and fix them. If appropriate, use sub-agents to investigate and fix each issue separately.
<file name="src/go/pkg/topology/engine/topology_adapter.go">
<violation number="1">
P1: Segment actors are always appended to the output regardless of identity-key deduplication. Unlike device and endpoint actors which `continue` (skip) when keys are empty or already present in `actorIndex`, this code only gates `addTopologyIdentityKeys` but not the `append`. This will produce duplicate actors when identity keys overlap.</violation>
</file>
Reply with feedback, questions, or to request a fix. Tag @cubic-dev-ai to re-run a review.
There was a problem hiding this comment.
1 issue found across 12 files (changes from recent commits).
Prompt for AI agents (all issues)
Check if these issues are valid — if so, understand the root cause of each and fix them. If appropriate, use sub-agents to investigate and fix each issue separately.
<file name="TODO-topology-library-phase2-direct-port.md">
<violation number="1">
P1: Do not commit real SNMP community strings in documentation. Replace `atadteN` with a redacted placeholder (e.g., `<community>`) everywhere it appears to avoid leaking credentials.</violation>
</file>
Reply with feedback, questions, or to request a fix. Tag @cubic-dev-ai to re-run a review.
| @@ -0,0 +1,2276 @@ | |||
| # TODO-topology-library-phase2-direct-port | |||
There was a problem hiding this comment.
P1: Do not commit real SNMP community strings in documentation. Replace atadteN with a redacted placeholder (e.g., <community>) everywhere it appears to avoid leaking credentials.
Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. At TODO-topology-library-phase2-direct-port.md:
<comment>Do not commit real SNMP community strings in documentation. Replace `atadteN` with a redacted placeholder (e.g., `<community>`) everywhere it appears to avoid leaking credentials.</comment>
There was a problem hiding this comment.
4 issues found across 34 files (changes from recent commits).
Prompt for AI agents (all issues)
Check if these issues are valid — if so, understand the root cause of each and fix them. If appropriate, use sub-agents to investigate and fix each issue separately.
<file name="src/go/plugin/go.d/collector/snmp/topology_cache.go">
<violation number="1">
P3: The first loop over `c.fdbEntries` (setting `vlanName`) is entirely redundant — the second loop immediately below iterates the same entries with the same filter and performs the same `vlanName` assignment in addition to setting `vlanID`. Remove the first loop to avoid the unnecessary O(n) pass.</violation>
</file>
<file name="src/go/pkg/topology/engine/topology_adapter.go">
<violation number="1">
P2: Inconsistent string literals: `HasPrefix` checks against `"macaddress:"` but slicing uses `len("macAddress:")`. While both are currently 11 bytes, this is a maintenance hazard — any future rename could silently break the offset. Use a single constant or the same literal for both, as the `management_ip:` case does.</violation>
<violation number="2">
P2: Missing overflow guard for `float64` → `int` conversion. The `int64` case properly checks `math.MaxInt`, but the `float64` case converts directly with `int(typed)`, which yields implementation-defined behavior for large values. Add a bounds check consistent with the `int64` branch.</violation>
</file>
<file name="src/go/plugin/go.d/config/go.d/snmp.profiles/default/_std-topology-cisco-vtp-mib.yaml">
<violation number="1">
P2: The vtpVersion scalar OID is missing the `.0` instance suffix, so the SNMP GET will not resolve and the vtp_version tag will be empty.</violation>
</file>
Reply with feedback, questions, or to request a fix. Tag @cubic-dev-ai to re-run a review.
| @@ -0,0 +1,3496 @@ | |||
| // SPDX-License-Identifier: GPL-3.0-or-later | |||
There was a problem hiding this comment.
P2: Missing overflow guard for float64 → int conversion. The int64 case properly checks math.MaxInt, but the float64 case converts directly with int(typed), which yields implementation-defined behavior for large values. Add a bounds check consistent with the int64 branch.
Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. At src/go/pkg/topology/engine/topology_adapter.go, line 3445:
<comment>Missing overflow guard for `float64` → `int` conversion. The `int64` case properly checks `math.MaxInt`, but the `float64` case converts directly with `int(typed)`, which yields implementation-defined behavior for large values. Add a bounds check consistent with the `int64` branch.</comment>
<file context>
@@ -1599,6 +2944,549 @@ func sortTopologyLinks(links []topology.Link) {
+ if typed > math.MaxInt {
+ return math.MaxInt
+ }
+ return int(typed)
+ case float64:
+ if typed <= 0 {
</file context>
| @@ -0,0 +1,38 @@ | |||
| # Supplemental topology profile for Cisco VTP VLAN metadata enrichment. | |||
There was a problem hiding this comment.
P2: The vtpVersion scalar OID is missing the .0 instance suffix, so the SNMP GET will not resolve and the vtp_version tag will be empty.
Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. At src/go/plugin/go.d/config/go.d/snmp.profiles/default/_std-topology-cisco-vtp-mib.yaml, line 7:
<comment>The vtpVersion scalar OID is missing the `.0` instance suffix, so the SNMP GET will not resolve and the vtp_version tag will be empty.</comment>
<file context>
@@ -0,0 +1,38 @@
+metric_tags:
+ - tag: vtp_version
+ symbol:
+ OID: 1.3.6.1.4.1.9.9.46.1.1.1
+ name: vtpVersion
+
</file context>
| @@ -0,0 +1,2987 @@ | |||
| // SPDX-License-Identifier: GPL-3.0-or-later | |||
There was a problem hiding this comment.
P3: The first loop over c.fdbEntries (setting vlanName) is entirely redundant — the second loop immediately below iterates the same entries with the same filter and performs the same vlanName assignment in addition to setting vlanID. Remove the first loop to avoid the unnecessary O(n) pass.
Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. At src/go/plugin/go.d/collector/snmp/topology_cache.go, line 1082:
<comment>The first loop over `c.fdbEntries` (setting `vlanName`) is entirely redundant — the second loop immediately below iterates the same entries with the same filter and performs the same `vlanName` assignment in addition to setting `vlanID`. Remove the first loop to avoid the unnecessary O(n) pass.</comment>
<file context>
@@ -906,29 +1016,200 @@ func (c *topologyCache) updateBridgePortMap(tags map[string]string) {
+ }
+
+ c.fdbIDToVlanID[fdbID] = vlanID
+ if vlanName := strings.TrimSpace(c.vlanIDToName[vlanID]); vlanName != "" {
+ for _, entry := range c.fdbEntries {
+ if entry == nil || strings.TrimSpace(entry.fdbID) != fdbID {
</file context>
There was a problem hiding this comment.
1 issue found across 10 files (changes from recent commits).
Prompt for AI agents (all issues)
Check if these issues are valid — if so, understand the root cause of each and fix them. If appropriate, use sub-agents to investigate and fix each issue separately.
<file name="src/go/pkg/topology/engine/topology_adapter.go">
<violation number="1">
P2: `"protocols"` and `"protocols_collected"` are set to the exact same value (`labelsCSVToSlice(dev.Labels, "protocols_observed")`). If they are meant to represent different concepts, one likely has the wrong source label key. If they are intentionally identical, this is redundant data that will increase payload size for every actor.</violation>
</file>
Reply with feedback, questions, or to request a fix. Tag @cubic-dev-ai to re-run a review.
7433fc6 to
36ab240
Compare
There was a problem hiding this comment.
2 issues found across 24 files (changes from recent commits).
Prompt for AI agents (unresolved issues)
Check if these issues are valid — if so, understand the root cause of each and fix them. If appropriate, use sub-agents to investigate and fix each issue separately.
<file name="src/go/tools/topology-oui-dataset/main.go">
<violation number="1">
P2: The CSV download uses http.DefaultClient with no timeout, so the CLI can hang indefinitely on stalled connections. Use a client with a reasonable timeout to avoid blocking the dataset generation.</violation>
</file>
<file name="TODO-topology-library-phase2-direct-port.md">
<violation number="1" location="TODO-topology-library-phase2-direct-port.md:186">
P1: Redact the SNMP community string from the documentation to avoid leaking credentials.
(Based on your team's feedback about redacting internal IP addresses and credentials.) [FEEDBACK_USED]</violation>
</file>
Reply with feedback, questions, or to request a fix. Tag @cubic-dev-ai to re-run a review.
| - Evidence collected: | ||
| - SNMP credentials/targets used from `/etc/netdata/go.d/snmp.conf`: | ||
| - devices: `10.20.4.1` (MikroTik-router), `10.20.4.3` (XS1930), `10.20.4.4` (GS1900), | ||
| - SNMP v2c community: `atadteN`. |
There was a problem hiding this comment.
P1: Redact the SNMP community string from the documentation to avoid leaking credentials.
(Based on your team's feedback about redacting internal IP addresses and credentials.)
Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. At TODO-topology-library-phase2-direct-port.md, line 186:
<comment>Redact the SNMP community string from the documentation to avoid leaking credentials.
(Based on your team's feedback about redacting internal IP addresses and credentials.) </comment>
<file context>
@@ -87,6 +107,190 @@
+ - Evidence collected:
+ - SNMP credentials/targets used from `/etc/netdata/go.d/snmp.conf`:
+ - devices: `10.20.4.1` (MikroTik-router), `10.20.4.3` (XS1930), `10.20.4.4` (GS1900),
+ - SNMP v2c community: `atadteN`.
+ - MikroTik-router (`10.20.4.1`) LLDP remote rows:
+ - `.1.0.8802.1.1.2.1.4.1.1.9.0.3.42 = "XS1930"`,
</file context>
| @@ -0,0 +1,198 @@ | |||
| // SPDX-License-Identifier: GPL-3.0-or-later | |||
There was a problem hiding this comment.
P2: The CSV download uses http.DefaultClient with no timeout, so the CLI can hang indefinitely on stalled connections. Use a client with a reasonable timeout to avoid blocking the dataset generation.
Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. At src/go/tools/topology-oui-dataset/main.go, line 101:
<comment>The CSV download uses http.DefaultClient with no timeout, so the CLI can hang indefinitely on stalled connections. Use a client with a reasonable timeout to avoid blocking the dataset generation.</comment>
<file context>
@@ -0,0 +1,198 @@
+ }
+ req.Header.Set("User-Agent", "netdata-topology-oui-dataset-updater/1.0")
+
+ resp, err := http.DefaultClient.Do(req)
+ if err != nil {
+ return nil, err
</file context>
There was a problem hiding this comment.
1 issue found across 9 files (changes from recent commits).
Prompt for AI agents (unresolved issues)
Check if these issues are valid — if so, understand the root cause of each and fix them. If appropriate, use sub-agents to investigate and fix each issue separately.
<file name="TODO-topology-library-phase2-direct-port.md">
<violation number="1" location="TODO-topology-library-phase2-direct-port.md:1404">
P1: Remove or redact the real internal IP addresses and SNMP community string from this TODO entry. These are sensitive environment details and credentials that should be replaced with placeholders or moved to a private runbook/config.
(Based on your team's feedback about redacting internal IPs and credentials.) [FEEDBACK_USED]</violation>
</file>
Reply with feedback, questions, or to request a fix. Tag @cubic-dev-ai to re-run a review.
| @@ -0,0 +1,1468 @@ | |||
| # TODO-topology-library-phase2-direct-port | |||
There was a problem hiding this comment.
P1: Remove or redact the real internal IP addresses and SNMP community string from this TODO entry. These are sensitive environment details and credentials that should be replaced with placeholders or moved to a private runbook/config.
(Based on your team's feedback about redacting internal IPs and credentials.)
Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. At TODO-topology-library-phase2-direct-port.md, line 1404:
<comment>Remove or redact the real internal IP addresses and SNMP community string from this TODO entry. These are sensitive environment details and credentials that should be replaced with placeholders or moved to a private runbook/config.
(Based on your team's feedback about redacting internal IPs and credentials.) </comment>
<file context>
@@ -1305,3 +1374,95 @@
+ - direct API calls with `map_type=lldp_cdp_managed|high_confidence_inferred|all_devices_low_confidence` all return `stats.map_type=lldp_cdp_managed`.
+ - this prevents exposing ARP/FDB-only endpoint links through the requested map mode and should be investigated separately.
+
+- [x] A36. Fix identity merge gaps for LLDP neighbors that also exist in ARP/ND (Costa, 2026-02-26).
+ - User constraint (Costa, 2026-02-26):
+ - ignore backward compatibility for this topology API path; choose the technically correct long-term model (UI is current sole consumer).
</file context>
This reverts commit bc58c2e.
# Conflicts: # src/go/plugin/agent/jobmgr/funcshandler.go
There was a problem hiding this comment.
3 issues found across 3 files (changes from recent commits).
Prompt for AI agents (unresolved issues)
Check if these issues are valid — if so, understand the root cause of each and fix them. If appropriate, use sub-agents to investigate and fix each issue separately.
<file name="src/collectors/log2journal/log2journal-logfmt.c">
<violation number="1">
P2: LOG_JOB is no longer initialized before parsing, so the hashtable and line key remain unset; logfmt_parse_document uses these fields via log_job_send_extracted_key_value, which can cause undefined behavior in this test path.</violation>
</file>
<file name="src/collectors/log2journal/log2journal-json.c">
<violation number="1">
P2: json_test bypasses log_job_init, leaving the job’s hashtable and line key uninitialized before json_parse_document writes extracted fields. This can crash in log_job_send_extracted_key_value. Reintroduce log_job_init (and prefix setup/cleanup) in the test path to keep the hashtable valid.</violation>
</file>
<file name="src/collectors/log2journal/log2journal-pcre2.c">
<violation number="1">
P2: pcre2_test no longer calls log_job_init, leaving the LOG_JOB hashtable and line key uninitialized. The subsequent log_job_send_extracted_key_value path assumes these are initialized, so this test can hit undefined behavior. Reintroduce log_job_init before creating the parser.</violation>
</file>
Reply with feedback, questions, or to request a fix. Tag @cubic-dev-ai to re-run a review.
There was a problem hiding this comment.
1 issue found across 17 files (changes from recent commits).
Prompt for AI agents (unresolved issues)
Check if these issues are valid — if so, understand the root cause of each and fix them. If appropriate, use sub-agents to investigate and fix each issue separately.
<file name="src/go/tools/topology-ip-intel-downloader/parse.go">
<violation number="1">
P2: Avoid converting the full payload to `string` before creating readers; it duplicates large datasets in memory. Use a `bytes.Reader`/`bytes.Buffer` over the `[]byte` to keep peak memory lower.</violation>
</file>
Reply with feedback, questions, or to request a fix. Tag @cubic-dev-ai to re-run a review.
| @@ -0,0 +1,329 @@ | |||
| // SPDX-License-Identifier: GPL-3.0-or-later | |||
There was a problem hiding this comment.
P2: Avoid converting the full payload to string before creating readers; it duplicates large datasets in memory. Use a bytes.Reader/bytes.Buffer over the []byte to keep peak memory lower.
Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. At src/go/tools/topology-ip-intel-downloader/parse.go, line 138:
<comment>Avoid converting the full payload to `string` before creating readers; it duplicates large datasets in memory. Use a `bytes.Reader`/`bytes.Buffer` over the `[]byte` to keep peak memory lower.</comment>
<file context>
@@ -0,0 +1,329 @@
+ asnRanges := make([]asnRange, 0, 1<<20)
+ countryRanges := make([]countryRange, 0, 1<<20)
+
+ scanner := bufio.NewScanner(strings.NewReader(string(payload)))
+ lineNo := 0
+ for scanner.Scan() {
</file context>
Summary
topologyandflows) for dashboard integrationMilestone 1: Data Collection ✅
This PR completes Milestone 1 — collecting topology and flow data at the agent level.
SNMP Topology (L2)
topologyfunction returning JSON with devices and linksNetFlow Collector (L3)
flowsfunction returning top talkers with bytes/packets/connectionsAggregation Tool
topology-flow-mergethat aggregates topology/flows from multiple Netdata agentsTesting
Roadmap
Test plan
go test ./plugin/go.d/collector/snmp/...— all LLDP/CDP tests passgo test ./plugin/go.d/collector/netflow/...— all flow decoder tests passSummary by cubic
Delivers multi-layer topology and flow analysis with a Rust NetFlow/IPFIX/sFlow plugin, a Go topology engine, and a journal-backed storage/query stack. Adds a live L7 network-connections topology and richer SNMP L2; removes the L3 SNMP pipeline.
New Features
Migration
Written for commit a5ae266. Summary will update on new commits.