Fix race condition for Failed Test Replay by daniel-mohedano · Pull Request #10679 · DataDog/dd-trace-java

daniel-mohedano · 2026-02-25T15:04:30Z

What Does This Do

Fix race condition in DebuggerSink.stop() that can cause DI snapshot loss during JVM shutdown, leading to flaky "test headless failed test replay" smoke tests.

During JVM shutdown, the periodic lowRateFlush on the dd-task-scheduler thread can race with the shutdown hook. The periodic flush drains snapshots from the BlockingQueue, but the thread's interrupt flag (set by the shutdown sequence) causes Moshi serialization to throw AssertionError: java.io.InterruptedIOException: interrupted. Since the snapshots have already been removed from the queue, they are permanently lost. The shutdown hook's subsequent flush then finds an empty queue and uploads nothing. This issue mostly surfaces in Failed Test Replay (and not Exception Replay) due to the nature of short-lived testing environments.

Evidence from CI logs of a failing run:

[dd-task-scheduler] DEBUG ...SnapshotSink - Sending snapshot for probe ...
[dd-task-scheduler] DEBUG ...AgentTaskScheduler - Uncaught exception from periodic task
java.lang.AssertionError: java.io.InterruptedIOException: interrupted
    at com.squareup.moshi.JsonAdapter.toJson(JsonAdapter.java:71)

The fix reorders stop() to cancel periodic schedules before performing the final flush. Additionally, SnapshotSink.getSerializedSnapshots() now re-queues snapshots that fail with an AssertionError so the shutdown hook flush can retry them with a non-interrupted thread. Normal serialization Exceptions still drop the snapshot.

The PR also introduces some additional changes to stabilize the smoke test environment:

Increases waitForLogs timeout from 15s to 30s for additional margin.
Set a very high DYNAMIC_INSTRUMENTATION_UPLOAD_FLUSH_INTERVAL value to minimize the race conditions.

Motivation

This issue was causing flakes in the FTR JUnit Console smoke test, where sometimes the snapshots were not available after test execution.

Contributor Checklist

Format the title according to the contribution guidelines
Assign the type: and (comp: or inst:) labels in addition to any other useful labels
Avoid using close, fix, or any linking keywords when referencing an issue
Use solves instead, and assign the PR milestone to the issue
Update the CODEOWNERS file on source file addition, migration, or deletion
Update public documentation with any new configuration flags or behaviors

Jira ticket: [PROJ-IDENT]

Note: Once your PR is ready to merge, add it to the merge queue by commenting /merge. /merge -c cancels the queue request. /merge -f --reason "reason" skips all merge queue checks; please use this judiciously, as some checks do not run at the PR-level. For more information, see this doc.

pr-commenter · 2026-02-25T15:22:51Z

Debugger benchmarks

Parameters

	Baseline	Candidate
baseline_or_candidate	baseline	candidate
ci_job_date	1772443491	1772443836
end_time	2026-03-02T09:26:11	2026-03-02T09:31:56
git_branch	master	daniel.mohedano/junit-console-flaky-fix
git_commit_sha	`b0ab971`	`0de9403`
start_time	2026-03-02T09:24:52	2026-03-02T09:30:37

See matching parameters

	Baseline	Candidate
ci_job_id	1466051197	1466051197
ci_pipeline_id	99698922	99698922
cpu_model	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
git_commit_date	1772442930	1772442930

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 10 metrics, 5 unstable metrics.

See unchanged results

scenario	Δ mean agg_http_req_duration_min	Δ mean agg_http_req_duration_p50	Δ mean agg_http_req_duration_p75	Δ mean agg_http_req_duration_p99	Δ mean throughput
scenario:noprobe	unstable [-14.260µs; +22.856µs] or [-5.310%; +8.512%]	unstable [-25.190µs; +32.446µs] or [-8.158%; +10.509%]	unstable [-34.521µs; +44.269µs] or [-10.703%; +13.725%]	unstable [-175.010µs; +44.059µs] or [-17.223%; +4.336%]	same
scenario:basic	unsure [+1.893µs; +9.096µs] or [+0.781%; +3.755%]	unsure [+2.147µs; +15.306µs] or [+0.803%; +5.726%]	unsure [+2.278µs; +16.535µs] or [+0.823%; +5.970%]	unstable [-140.420µs; +34.395µs] or [-17.543%; +4.297%]	same
scenario:loop	unsure [+6.216µs; +13.259µs] or [+0.070%; +0.150%]	unsure [+4.330µs; +14.358µs] or [+0.048%; +0.160%]	unsure [+0.743µs; +12.454µs] or [+0.008%; +0.138%]	same	same

Request duration reports for reports

gantt
    title reports - request duration [CI 0.99] : candidate=None, baseline=None
    dateFormat X
    axisFormat %s
section baseline
noprobe (308.756 µs) : 284, 333
.   : milestone, 309,
basic (267.282 µs) : 261, 274
.   : milestone, 267,
loop (8.957 ms) : 8953, 8962
.   : milestone, 8957,
section candidate
noprobe (312.384 µs) : 282, 342
.   : milestone, 312,
basic (276.009 µs) : 269, 283
.   : milestone, 276,
loop (8.967 ms) : 8962, 8972
.   : milestone, 8967,

baseline results

Scenario	Request median duration [CI 0.99]
noprobe	308.756 µs [284.221 µs, 333.291 µs]
basic	267.282 µs [261.056 µs, 273.509 µs]
loop	8.957 ms [8.953 ms, 8.962 ms]

candidate results

Scenario	Request median duration [CI 0.99]
noprobe	312.384 µs [282.314 µs, 342.454 µs]
basic	276.009 µs [269.324 µs, 282.694 µs]
loop	8.967 ms [8.962 ms, 8.972 ms]

pr-commenter · 2026-02-25T15:50:55Z

Benchmarks

Startup

Parameters

	Baseline	Candidate
baseline_or_candidate	baseline	candidate
git_branch	master	daniel.mohedano/junit-console-flaky-fix
git_commit_date	1772065756	1772442930
git_commit_sha	`969d21d`	`0de9403`
release_version	1.60.0-SNAPSHOT~969d21d507	1.60.0-SNAPSHOT~0de9403e81

See matching parameters

	Baseline	Candidate
application	insecure-bank	insecure-bank
ci_job_date	1772444805	1772444805
ci_job_id	1466051183	1466051183
ci_pipeline_id	99698922	99698922
cpu_model	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version	Linux runner-zfyrx7zua-project-304-concurrent-0-9kaj3afx 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux	Linux runner-zfyrx7zua-project-304-concurrent-0-9kaj3afx 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux
module	Agent	Agent
parent	None	None

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 63 metrics, 8 unstable metrics.

Startup time reports for petclinic

gantt
    title petclinic - global startup overhead: candidate=1.60.0-SNAPSHOT~0de9403e81, baseline=1.60.0-SNAPSHOT~969d21d507

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.065 s) : 0, 1064688
Total [baseline] (10.936 s) : 0, 10935988
Agent [candidate] (1.065 s) : 0, 1064522
Total [candidate] (10.937 s) : 0, 10937217
section appsec
Agent [baseline] (1.238 s) : 0, 1238318
Total [baseline] (10.985 s) : 0, 10984873
Agent [candidate] (1.238 s) : 0, 1238156
Total [candidate] (11.086 s) : 0, 11085515
section iast
Agent [baseline] (1.239 s) : 0, 1239381
Total [baseline] (11.262 s) : 0, 11261833
Agent [candidate] (1.24 s) : 0, 1239509
Total [candidate] (11.143 s) : 0, 11143021
section profiling
Agent [baseline] (1.197 s) : 0, 1196553
Total [baseline] (10.969 s) : 0, 10969143
Agent [candidate] (1.193 s) : 0, 1193027
Total [candidate] (10.954 s) : 0, 10953969

baseline results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.065 s	-
Agent	appsec	1.238 s	173.631 ms (16.3%)
Agent	iast	1.239 s	174.693 ms (16.4%)
Agent	profiling	1.197 s	131.866 ms (12.4%)
Total	tracing	10.936 s	-
Total	appsec	10.985 s	48.885 ms (0.4%)
Total	iast	11.262 s	325.845 ms (3.0%)
Total	profiling	10.969 s	33.155 ms (0.3%)

candidate results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.065 s	-
Agent	appsec	1.238 s	173.634 ms (16.3%)
Agent	iast	1.24 s	174.987 ms (16.4%)
Agent	profiling	1.193 s	128.506 ms (12.1%)
Total	tracing	10.937 s	-
Total	appsec	11.086 s	148.298 ms (1.4%)
Total	iast	11.143 s	205.803 ms (1.9%)
Total	profiling	10.954 s	16.752 ms (0.2%)

gantt
    title petclinic - break down per module: candidate=1.60.0-SNAPSHOT~0de9403e81, baseline=1.60.0-SNAPSHOT~969d21d507

    dateFormat X
    axisFormat %s
section tracing
crashtracking [baseline] (1.183 ms) : 0, 1183
crashtracking [candidate] (1.178 ms) : 0, 1178
BytebuddyAgent [baseline] (627.123 ms) : 0, 627123
BytebuddyAgent [candidate] (627.098 ms) : 0, 627098
AgentMeter [baseline] (29.108 ms) : 0, 29108
AgentMeter [candidate] (29.149 ms) : 0, 29149
GlobalTracer [baseline] (256.864 ms) : 0, 256864
GlobalTracer [candidate] (257.468 ms) : 0, 257468
AppSec [baseline] (33.069 ms) : 0, 33069
AppSec [candidate] (32.903 ms) : 0, 32903
Debugger [baseline] (66.287 ms) : 0, 66287
Debugger [candidate] (65.591 ms) : 0, 65591
Remote Config [baseline] (609.482 µs) : 0, 609
Remote Config [candidate] (607.379 µs) : 0, 607
Telemetry [baseline] (10.603 ms) : 0, 10603
Telemetry [candidate] (9.961 ms) : 0, 9961
Flare Poller [baseline] (3.829 ms) : 0, 3829
Flare Poller [candidate] (4.544 ms) : 0, 4544
section appsec
crashtracking [baseline] (1.184 ms) : 0, 1184
crashtracking [candidate] (1.177 ms) : 0, 1177
BytebuddyAgent [baseline] (657.991 ms) : 0, 657991
BytebuddyAgent [candidate] (657.492 ms) : 0, 657492
AgentMeter [baseline] (11.913 ms) : 0, 11913
AgentMeter [candidate] (11.933 ms) : 0, 11933
GlobalTracer [baseline] (257.887 ms) : 0, 257887
GlobalTracer [candidate] (258.196 ms) : 0, 258196
IAST [baseline] (25.198 ms) : 0, 25198
IAST [candidate] (25.276 ms) : 0, 25276
AppSec [baseline] (167.641 ms) : 0, 167641
AppSec [candidate] (167.441 ms) : 0, 167441
Debugger [baseline] (66.876 ms) : 0, 66876
Debugger [candidate] (66.805 ms) : 0, 66805
Remote Config [baseline] (646.372 µs) : 0, 646
Remote Config [candidate] (665.254 µs) : 0, 665
Telemetry [baseline] (9.333 ms) : 0, 9333
Telemetry [candidate] (9.479 ms) : 0, 9479
Flare Poller [baseline] (3.703 ms) : 0, 3703
Flare Poller [candidate] (3.695 ms) : 0, 3695
section iast
crashtracking [baseline] (1.192 ms) : 0, 1192
crashtracking [candidate] (1.189 ms) : 0, 1189
BytebuddyAgent [baseline] (801.024 ms) : 0, 801024
BytebuddyAgent [candidate] (799.827 ms) : 0, 799827
AgentMeter [baseline] (11.363 ms) : 0, 11363
AgentMeter [candidate] (11.403 ms) : 0, 11403
GlobalTracer [baseline] (248.218 ms) : 0, 248218
GlobalTracer [candidate] (249.532 ms) : 0, 249532
IAST [baseline] (27.441 ms) : 0, 27441
IAST [candidate] (27.359 ms) : 0, 27359
AppSec [baseline] (34.896 ms) : 0, 34896
AppSec [candidate] (34.159 ms) : 0, 34159
Debugger [baseline] (66.358 ms) : 0, 66358
Debugger [candidate] (67.357 ms) : 0, 67357
Remote Config [baseline] (538.552 µs) : 0, 539
Remote Config [candidate] (540.716 µs) : 0, 541
Telemetry [baseline] (8.673 ms) : 0, 8673
Telemetry [candidate] (8.627 ms) : 0, 8627
Flare Poller [baseline] (3.489 ms) : 0, 3489
Flare Poller [candidate] (3.468 ms) : 0, 3468
section profiling
crashtracking [baseline] (1.171 ms) : 0, 1171
crashtracking [candidate] (1.176 ms) : 0, 1176
BytebuddyAgent [baseline] (686.222 ms) : 0, 686222
BytebuddyAgent [candidate] (683.59 ms) : 0, 683590
AgentMeter [baseline] (8.59 ms) : 0, 8590
AgentMeter [candidate] (8.558 ms) : 0, 8558
GlobalTracer [baseline] (216.835 ms) : 0, 216835
GlobalTracer [candidate] (216.526 ms) : 0, 216526
AppSec [baseline] (32.861 ms) : 0, 32861
AppSec [candidate] (32.541 ms) : 0, 32541
Debugger [baseline] (67.504 ms) : 0, 67504
Debugger [candidate] (67.283 ms) : 0, 67283
Remote Config [baseline] (630.065 µs) : 0, 630
Remote Config [candidate] (617.85 µs) : 0, 618
Telemetry [baseline] (8.9 ms) : 0, 8900
Telemetry [candidate] (9.042 ms) : 0, 9042
Flare Poller [baseline] (3.76 ms) : 0, 3760
Flare Poller [candidate] (3.747 ms) : 0, 3747
ProfilingAgent [baseline] (99.022 ms) : 0, 99022
ProfilingAgent [candidate] (99.252 ms) : 0, 99252
Profiling [baseline] (99.589 ms) : 0, 99589
Profiling [candidate] (99.82 ms) : 0, 99820

Startup time reports for insecure-bank

gantt
    title insecure-bank - global startup overhead: candidate=1.60.0-SNAPSHOT~0de9403e81, baseline=1.60.0-SNAPSHOT~969d21d507

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.063 s) : 0, 1063341
Total [baseline] (8.728 s) : 0, 8728009
Agent [candidate] (1.065 s) : 0, 1064507
Total [candidate] (8.803 s) : 0, 8803020
section iast
Agent [baseline] (1.229 s) : 0, 1229244
Total [baseline] (9.369 s) : 0, 9369413
Agent [candidate] (1.232 s) : 0, 1231667
Total [candidate] (9.374 s) : 0, 9374123

baseline results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.063 s	-
Agent	iast	1.229 s	165.904 ms (15.6%)
Total	tracing	8.728 s	-
Total	iast	9.369 s	641.404 ms (7.3%)

candidate results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.065 s	-
Agent	iast	1.232 s	167.161 ms (15.7%)
Total	tracing	8.803 s	-
Total	iast	9.374 s	571.103 ms (6.5%)

gantt
    title insecure-bank - break down per module: candidate=1.60.0-SNAPSHOT~0de9403e81, baseline=1.60.0-SNAPSHOT~969d21d507

    dateFormat X
    axisFormat %s
section tracing
crashtracking [baseline] (1.186 ms) : 0, 1186
crashtracking [candidate] (1.184 ms) : 0, 1184
BytebuddyAgent [baseline] (627.223 ms) : 0, 627223
BytebuddyAgent [candidate] (627.224 ms) : 0, 627224
AgentMeter [baseline] (29.127 ms) : 0, 29127
AgentMeter [candidate] (29.138 ms) : 0, 29138
GlobalTracer [baseline] (257.283 ms) : 0, 257283
GlobalTracer [candidate] (258.14 ms) : 0, 258140
AppSec [baseline] (33.034 ms) : 0, 33034
AppSec [candidate] (32.812 ms) : 0, 32812
Debugger [baseline] (62.025 ms) : 0, 62025
Debugger [candidate] (64.785 ms) : 0, 64785
Remote Config [baseline] (620.882 µs) : 0, 621
Remote Config [candidate] (617.798 µs) : 0, 618
Telemetry [baseline] (9.841 ms) : 0, 9841
Telemetry [candidate] (9.12 ms) : 0, 9120
Flare Poller [baseline] (6.907 ms) : 0, 6907
Flare Poller [candidate] (5.336 ms) : 0, 5336
section iast
crashtracking [baseline] (1.185 ms) : 0, 1185
crashtracking [candidate] (1.19 ms) : 0, 1190
BytebuddyAgent [baseline] (794.302 ms) : 0, 794302
BytebuddyAgent [candidate] (797.628 ms) : 0, 797628
AgentMeter [baseline] (11.292 ms) : 0, 11292
AgentMeter [candidate] (11.257 ms) : 0, 11257
GlobalTracer [baseline] (246.545 ms) : 0, 246545
GlobalTracer [candidate] (246.678 ms) : 0, 246678
IAST [baseline] (27.078 ms) : 0, 27078
IAST [candidate] (26.783 ms) : 0, 26783
AppSec [baseline] (35.03 ms) : 0, 35030
AppSec [candidate] (33.976 ms) : 0, 33976
Debugger [baseline] (65.006 ms) : 0, 65006
Debugger [candidate] (65.397 ms) : 0, 65397
Remote Config [baseline] (547.632 µs) : 0, 548
Remote Config [candidate] (532.39 µs) : 0, 532
Telemetry [baseline] (8.714 ms) : 0, 8714
Telemetry [candidate] (8.639 ms) : 0, 8639
Flare Poller [baseline] (3.507 ms) : 0, 3507
Flare Poller [candidate] (3.508 ms) : 0, 3508

Load

Parameters

	Baseline	Candidate
baseline_or_candidate	baseline	candidate
git_branch	master	daniel.mohedano/junit-console-flaky-fix
git_commit_date	1772065756	1772442930
git_commit_sha	`969d21d`	`0de9403`
release_version	1.60.0-SNAPSHOT~969d21d507	1.60.0-SNAPSHOT~0de9403e81

See matching parameters

	Baseline	Candidate
application	insecure-bank	insecure-bank
ci_job_date	1772445210	1772445210
ci_job_id	1466051185	1466051185
ci_pipeline_id	99698922	99698922
cpu_model	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version	Linux runner-zfyrx7zua-project-304-concurrent-0-qdc4dkyo 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux	Linux runner-zfyrx7zua-project-304-concurrent-0-qdc4dkyo 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux

Summary

Found 1 performance improvements and 4 performance regressions! Performance is the same for 9 metrics, 22 unstable metrics.

scenario	Δ mean agg_http_req_duration_p50	Δ mean agg_http_req_duration_p95	Δ mean throughput	candidate mean agg_http_req_duration_p50	candidate mean agg_http_req_duration_p95	candidate mean throughput	baseline mean agg_http_req_duration_p50	baseline mean agg_http_req_duration_p95	baseline mean throughput
scenario:load:insecure-bank:profiling:high_load	worse [+37.443µs; +151.260µs] or [+2.329%; +9.408%]	unstable [-229.649µs; +681.586µs] or [-4.984%; +14.792%]	unstable [-457.951op/s; +143.514op/s] or [-20.030%; +6.277%]	1.702ms	4.834ms	2129.062op/s	1.608ms	4.608ms	2286.281op/s
scenario:load:petclinic:profiling:high_load	worse [+1.983ms; +2.606ms] or [+10.937%; +14.376%]	worse [+1.287ms; +3.037ms] or [+4.250%; +10.032%]	unstable [-50.273op/s; -0.289op/s] or [-20.072%; -0.115%]	20.423ms	32.433ms	225.188op/s	18.129ms	30.272ms	250.469op/s
scenario:load:petclinic:appsec:high_load	better [-1.654ms; -0.978ms] or [-8.527%; -5.042%]	unsure [-1.830ms; -0.466ms] or [-5.915%; -1.507%]	unstable [-11.140op/s; +39.453op/s] or [-4.683%; +16.586%]	18.081ms	29.786ms	252.031op/s	19.397ms	30.934ms	237.875op/s
scenario:load:petclinic:no_agent:high_load	worse [+0.494ms; +1.969ms] or [+2.813%; +11.222%]	unstable [+0.527ms; +3.697ms] or [+1.794%; +12.592%]	unstable [-46.201op/s; +9.951op/s] or [-17.830%; +3.840%]	18.775ms	31.468ms	241.000op/s	17.544ms	29.356ms	259.125op/s

Request duration reports for insecure-bank

gantt
    title insecure-bank - request duration [CI 0.99] : candidate=1.60.0-SNAPSHOT~0de9403e81, baseline=1.60.0-SNAPSHOT~969d21d507
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.194 ms) : 1182, 1205
.   : milestone, 1194,
iast (3.116 ms) : 3074, 3158
.   : milestone, 3116,
iast_FULL (5.737 ms) : 5680, 5793
.   : milestone, 5737,
iast_GLOBAL (3.619 ms) : 3558, 3681
.   : milestone, 3619,
profiling (1.973 ms) : 1955, 1991
.   : milestone, 1973,
tracing (1.836 ms) : 1820, 1852
.   : milestone, 1836,
section candidate
no_agent (1.165 ms) : 1154, 1176
.   : milestone, 1165,
iast (3.343 ms) : 3298, 3388
.   : milestone, 3343,
iast_FULL (5.996 ms) : 5936, 6057
.   : milestone, 5996,
iast_GLOBAL (3.661 ms) : 3605, 3718
.   : milestone, 3661,
profiling (2.123 ms) : 2101, 2145
.   : milestone, 2123,
tracing (1.802 ms) : 1787, 1817
.   : milestone, 1802,

baseline results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	1.194 ms [1.182 ms, 1.205 ms]	-
iast	3.116 ms [3.074 ms, 3.158 ms]	1.922 ms (161.0%)
iast_FULL	5.737 ms [5.68 ms, 5.793 ms]	4.543 ms (380.5%)
iast_GLOBAL	3.619 ms [3.558 ms, 3.681 ms]	2.426 ms (203.2%)
profiling	1.973 ms [1.955 ms, 1.991 ms]	779.367 µs (65.3%)
tracing	1.836 ms [1.82 ms, 1.852 ms]	641.985 µs (53.8%)

candidate results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	1.165 ms [1.154 ms, 1.176 ms]	-
iast	3.343 ms [3.298 ms, 3.388 ms]	2.178 ms (187.0%)
iast_FULL	5.996 ms [5.936 ms, 6.057 ms]	4.832 ms (414.8%)
iast_GLOBAL	3.661 ms [3.605 ms, 3.718 ms]	2.496 ms (214.3%)
profiling	2.123 ms [2.101 ms, 2.145 ms]	957.999 µs (82.2%)
tracing	1.802 ms [1.787 ms, 1.817 ms]	636.993 µs (54.7%)

Request duration reports for petclinic

gantt
    title petclinic - request duration [CI 0.99] : candidate=1.60.0-SNAPSHOT~0de9403e81, baseline=1.60.0-SNAPSHOT~969d21d507
    dateFormat X
    axisFormat %s
section baseline
no_agent (18.009 ms) : 17820, 18197
.   : milestone, 18009,
appsec (19.624 ms) : 19428, 19820
.   : milestone, 19624,
code_origins (17.642 ms) : 17467, 17817
.   : milestone, 17642,
iast (17.517 ms) : 17343, 17692
.   : milestone, 17517,
profiling (18.635 ms) : 18446, 18823
.   : milestone, 18635,
tracing (17.749 ms) : 17573, 17926
.   : milestone, 17749,
section candidate
no_agent (19.376 ms) : 19177, 19574
.   : milestone, 19376,
appsec (18.516 ms) : 18330, 18702
.   : milestone, 18516,
code_origins (17.856 ms) : 17679, 18032
.   : milestone, 17856,
iast (17.678 ms) : 17501, 17855
.   : milestone, 17678,
profiling (20.73 ms) : 20522, 20938
.   : milestone, 20730,
tracing (17.409 ms) : 17238, 17579
.   : milestone, 17409,

baseline results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	18.009 ms [17.82 ms, 18.197 ms]	-
appsec	19.624 ms [19.428 ms, 19.82 ms]	1.615 ms (9.0%)
code_origins	17.642 ms [17.467 ms, 17.817 ms]	-367.235 µs (-2.0%)
iast	17.517 ms [17.343 ms, 17.692 ms]	-491.687 µs (-2.7%)
profiling	18.635 ms [18.446 ms, 18.823 ms]	625.729 µs (3.5%)
tracing	17.749 ms [17.573 ms, 17.926 ms]	-259.874 µs (-1.4%)

candidate results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	19.376 ms [19.177 ms, 19.574 ms]	-
appsec	18.516 ms [18.33 ms, 18.702 ms]	-860.084 µs (-4.4%)
code_origins	17.856 ms [17.679 ms, 18.032 ms]	-1.52 ms (-7.8%)
iast	17.678 ms [17.501 ms, 17.855 ms]	-1.698 ms (-8.8%)
profiling	20.73 ms [20.522 ms, 20.938 ms]	1.354 ms (7.0%)
tracing	17.409 ms [17.238 ms, 17.579 ms]	-1.967 ms (-10.2%)

Dacapo

Parameters

	Baseline	Candidate
baseline_or_candidate	baseline	candidate
git_branch	master	daniel.mohedano/junit-console-flaky-fix
git_commit_date	1772065756	1772442930
git_commit_sha	`969d21d`	`0de9403`
release_version	1.60.0-SNAPSHOT~969d21d507	1.60.0-SNAPSHOT~0de9403e81

See matching parameters

	Baseline	Candidate
application	biojava	biojava
ci_job_date	1772444992	1772444992
ci_job_id	1466051186	1466051186
ci_pipeline_id	99698922	99698922
cpu_model	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version	Linux runner-zfyrx7zua-project-304-concurrent-0-77b7x2fp 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux	Linux runner-zfyrx7zua-project-304-concurrent-0-77b7x2fp 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 11 metrics, 1 unstable metrics.

Execution time for biojava

gantt
    title biojava - execution time [CI 0.99] : candidate=1.60.0-SNAPSHOT~0de9403e81, baseline=1.60.0-SNAPSHOT~969d21d507
    dateFormat X
    axisFormat %s
section baseline
no_agent (15.341 s) : 15341000, 15341000
.   : milestone, 15341000,
appsec (14.905 s) : 14905000, 14905000
.   : milestone, 14905000,
iast (18.211 s) : 18211000, 18211000
.   : milestone, 18211000,
iast_GLOBAL (17.974 s) : 17974000, 17974000
.   : milestone, 17974000,
profiling (14.709 s) : 14709000, 14709000
.   : milestone, 14709000,
tracing (14.669 s) : 14669000, 14669000
.   : milestone, 14669000,
section candidate
no_agent (14.743 s) : 14743000, 14743000
.   : milestone, 14743000,
appsec (14.688 s) : 14688000, 14688000
.   : milestone, 14688000,
iast (18.228 s) : 18228000, 18228000
.   : milestone, 18228000,
iast_GLOBAL (17.773 s) : 17773000, 17773000
.   : milestone, 17773000,
profiling (14.964 s) : 14964000, 14964000
.   : milestone, 14964000,
tracing (15.102 s) : 15102000, 15102000
.   : milestone, 15102000,

baseline results

Variant	Execution Time [CI 0.99]	Δ no_agent
no_agent	15.341 s [15.341 s, 15.341 s]	-
appsec	14.905 s [14.905 s, 14.905 s]	-436.0 ms (-2.8%)
iast	18.211 s [18.211 s, 18.211 s]	2.87 s (18.7%)
iast_GLOBAL	17.974 s [17.974 s, 17.974 s]	2.633 s (17.2%)
profiling	14.709 s [14.709 s, 14.709 s]	-632.0 ms (-4.1%)
tracing	14.669 s [14.669 s, 14.669 s]	-672.0 ms (-4.4%)

candidate results

Variant	Execution Time [CI 0.99]	Δ no_agent
no_agent	14.743 s [14.743 s, 14.743 s]	-
appsec	14.688 s [14.688 s, 14.688 s]	-55.0 ms (-0.4%)
iast	18.228 s [18.228 s, 18.228 s]	3.485 s (23.6%)
iast_GLOBAL	17.773 s [17.773 s, 17.773 s]	3.03 s (20.6%)
profiling	14.964 s [14.964 s, 14.964 s]	221.0 ms (1.5%)
tracing	15.102 s [15.102 s, 15.102 s]	359.0 ms (2.4%)

Execution time for tomcat

gantt
    title tomcat - execution time [CI 0.99] : candidate=1.60.0-SNAPSHOT~0de9403e81, baseline=1.60.0-SNAPSHOT~969d21d507
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.472 ms) : 1461, 1484
.   : milestone, 1472,
appsec (3.802 ms) : 3579, 4025
.   : milestone, 3802,
iast (2.25 ms) : 2182, 2319
.   : milestone, 2250,
iast_GLOBAL (2.293 ms) : 2224, 2362
.   : milestone, 2293,
profiling (2.1 ms) : 2043, 2156
.   : milestone, 2100,
tracing (2.057 ms) : 2004, 2110
.   : milestone, 2057,
section candidate
no_agent (1.468 ms) : 1457, 1480
.   : milestone, 1468,
appsec (3.783 ms) : 3562, 4005
.   : milestone, 3783,
iast (2.256 ms) : 2187, 2325
.   : milestone, 2256,
iast_GLOBAL (2.293 ms) : 2224, 2361
.   : milestone, 2293,
profiling (2.12 ms) : 2064, 2177
.   : milestone, 2120,
tracing (2.059 ms) : 2006, 2112
.   : milestone, 2059,

baseline results

Variant	Execution Time [CI 0.99]	Δ no_agent
no_agent	1.472 ms [1.461 ms, 1.484 ms]	-
appsec	3.802 ms [3.579 ms, 4.025 ms]	2.33 ms (158.3%)
iast	2.25 ms [2.182 ms, 2.319 ms]	778.032 µs (52.9%)
iast_GLOBAL	2.293 ms [2.224 ms, 2.362 ms]	820.765 µs (55.8%)
profiling	2.1 ms [2.043 ms, 2.156 ms]	627.472 µs (42.6%)
tracing	2.057 ms [2.004 ms, 2.11 ms]	584.805 µs (39.7%)

candidate results

Variant	Execution Time [CI 0.99]	Δ no_agent
no_agent	1.468 ms [1.457 ms, 1.48 ms]	-
appsec	3.783 ms [3.562 ms, 4.005 ms]	2.315 ms (157.7%)
iast	2.256 ms [2.187 ms, 2.325 ms]	787.902 µs (53.7%)
iast_GLOBAL	2.293 ms [2.224 ms, 2.361 ms]	824.341 µs (56.1%)
profiling	2.12 ms [2.064 ms, 2.177 ms]	652.08 µs (44.4%)
tracing	2.059 ms [2.006 ms, 2.112 ms]	590.606 µs (40.2%)

github-actions · 2026-02-26T12:32:34Z

Hi! 👋 Thanks for your pull request! 🎉

To help us review it, please make sure to:

Add at least one type, and one component or instrumentation label to the pull request

If you need help, please check our contributing guidelines.

amarziali · 2026-02-26T12:34:35Z

dd-java-agent/agent-debugger/src/main/java/com/datadog/debugger/sink/DebuggerSink.java

    cancelSchedule(this.lowRateScheduled);
+    // clear interrupt flag that could be set by JVM shutdown to allow to serialize and upload
+    // snapshots
+    Thread.interrupted();


Is there a catch for InterruptedException that is not reinterrupting the thread? Since I'm not sure that putting this here is the best option

In hindsight I realize that the interruption is not even in the thread performing the stop method, but rather the one with the scheduled flush, so I don't think the interrupted call does anything here, I'll update the approach (and the InterruptedException can be caught in

dd-trace-java/dd-java-agent/agent-debugger/src/main/java/com/datadog/debugger/sink/SnapshotSink.java

Line 171 in 0007547

private List<String> getSerializedSnapshots(BlockingQueue<Snapshot> queue, int localBatchSize) {

)

amarziali

Looks OK to me. I leave DI team give the formal approval. Thanks for the changes!

jpbempel · 2026-03-02T08:36:25Z

dd-java-agent/agent-debugger/src/main/java/com/datadog/debugger/sink/SnapshotSink.java

        }
      } catch (Exception e) {
        ExceptionHelper.logException(LOGGER, e, "Error during snapshot serialization:");
+      } catch (Throwable e) {


Can we target more specifically Errors here?
If Moshi is raising AssertionError, let's focus on it.
Otherwise, catching Throwable here means we are catching OutOfMemoryError which I doubt is good idea there

Good point, addressed in 0de9403 to reduce the scope to only AssertionError

fix: flakiness in junit console FTR test

6f28890

daniel-mohedano added 3 commits February 26, 2026 09:43

chore: update comment

60a8235

Merge branch 'master' into daniel.mohedano/junit-console-flaky-fix

ccae7d4

fix: also update flush interval

aba284a

daniel-mohedano marked this pull request as ready for review February 26, 2026 12:32

daniel-mohedano requested review from a team as code owners February 26, 2026 12:32

daniel-mohedano requested review from amarziali and tylfin and removed request for a team February 26, 2026 12:32

daniel-mohedano added type: bug Bug report and fix comp: debugger Dynamic Instrumentation labels Feb 26, 2026

tylfin requested a review from jpbempel February 26, 2026 12:34

amarziali reviewed Feb 26, 2026

View reviewed changes

fix: update approach to requeue

aff07bd

amarziali reviewed Feb 26, 2026

View reviewed changes

jpbempel reviewed Mar 2, 2026

View reviewed changes

feat: reduce catch scope to assertionerror

0de9403

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Fix race condition for Failed Test Replay#10679

Fix race condition for Failed Test Replay#10679
daniel-mohedano wants to merge 6 commits intomasterfrom
daniel.mohedano/junit-console-flaky-fix

daniel-mohedano commented Feb 25, 2026 •

edited

Loading

Uh oh!

pr-commenter bot commented Feb 25, 2026 •

edited

Loading

Uh oh!

pr-commenter bot commented Feb 25, 2026 •

edited

Loading

Uh oh!

github-actions bot commented Feb 26, 2026

Uh oh!

amarziali Feb 26, 2026

Uh oh!

daniel-mohedano Feb 26, 2026

Uh oh!

amarziali left a comment

Uh oh!

jpbempel Mar 2, 2026

Uh oh!

daniel-mohedano Mar 2, 2026 •

edited

Loading

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Conversation

daniel-mohedano commented Feb 25, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What Does This Do

Motivation

Contributor Checklist

Uh oh!

pr-commenter bot commented Feb 25, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Debugger benchmarks

Parameters

Summary

Uh oh!

pr-commenter bot commented Feb 25, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Benchmarks

Startup

Parameters

Summary

Load

Parameters

Summary

Dacapo

Parameters

Summary

Uh oh!

github-actions bot commented Feb 26, 2026

Uh oh!

amarziali Feb 26, 2026

Choose a reason for hiding this comment

Uh oh!

daniel-mohedano Feb 26, 2026

Choose a reason for hiding this comment

Uh oh!

amarziali left a comment

Choose a reason for hiding this comment

Uh oh!

jpbempel Mar 2, 2026

Choose a reason for hiding this comment

Uh oh!

daniel-mohedano Mar 2, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

daniel-mohedano commented Feb 25, 2026 •

edited

Loading

pr-commenter bot commented Feb 25, 2026 •

edited

Loading

pr-commenter bot commented Feb 25, 2026 •

edited

Loading

daniel-mohedano Mar 2, 2026 •

edited

Loading