Track 7 KPIs: MTTD, MTTR(-C), Precision/Recall, Noise Index, Coverage, Response Success Rate, and Ops & Collaboration Quality (SLA/reporting).
Benchmarks (typical enterprise targets):
MTTD: minutes—< 15–30 min for top-tier
MTTR(-C): < 60 min to contain/respond (shorter for high‑risk)
Precision/Recall: >80% / >90% in PoC scenarios
Noise Index: operator‑digestible; e.g., < 10 alerts/day per 1,000 EP
Coverage: >90% key assets onboarded; >80% priority TTP coverage
Response Success: >90% of high‑risk incidents contained/eradicated
Ops Quality: SLA ≥95%, <15 min first response; IOC + timeline + clear actions in every case note
KPI 1 — MTTD (Mean Time To Detect)
What it means: Time from threat occurrence to first MDR detection/flag.
How to measure: Difference between the attack simulation timestamp (MITRE ATT&CK‑based; e.g., ransomware initial access, credential dump) and the first detection alert time.
Benchmark: Minutes to tens of minutes; top‑tier < 15–30 min.
Pro tip: Record both UTC and local timestamps; correct for data ingestion latency.
KPI 2 — MTTR‑C / MTTR (Contain / Respond)
What it means: Time from detection to containment/blocking and actionable guidance completed.
How to measure: First detection alert → endpoint isolation / account disable / IOC block (or playbook action) completed.
Benchmark: < 60 min (tighter for high‑risk); automation shortens this materially.
Pro tip: Separate acknowledge, contain, and eradicate to see where time is spent.
KPI 3 — Detection Quality (Precision & Recall)
Precision = TP / (TP + FP): share of alerts that were real → reduces waste.
Recall = TP / (TP + FN): share of real threats caught → reduces misses.
How to measure: Pre‑agreed red‑team/ATT&CK emulation scenarios; label ground truth.
Benchmark: PoC goal Precision >80%, Recall >90% (tune by environment).
Pro tip: Track per‑scenario metrics (e.g., C2 beacon vs. LOLBins) to avoid averages hiding gaps.
KPI 4 — Noise Index (False Positives & Alert Load)
What it means: Volume and ratio of “unnecessary alerts” over day/week.
How to measure: FP/day, FP%, and analyst‑actionable alerts per 100 endpoints/day.
Benchmark: Operator‑digestible; e.g., < 10/day per 1,000 endpoints.
Pro tip: Require vendors to turn on deduplication and suppression they’d recommend in production.
KPI 5 — Coverage (Visibility & ATT&CK Technique Breadth)
What it means: Which telemetry/logs are collected (EPP/EDR, IDP, email, cloud, network) and which ATT&CK TTPs are detectable.
How to measure: % asset/platform onboarding + % ATT&CK T/TA covered + data latency.
Benchmark: >90% of critical assets onboarded; >80% of priority TTPs (ransomware, credential abuse, Living‑off‑the‑Land, data exfiltration).
Pro tip: List required connectors by “Required / Optional” and verify ingest health.
KPI 6 — Response Success Rate
What it means: Share of high‑risk incidents where risk was actually reduced (contain/isolate/block and/or customer‑executed recommendation).
How to measure: Count success vs. misses across high‑risk cases; if MDR is advisory‑only, include customer action rate.
Benchmark: >90% within PoC scope.
Pro tip: Ask for per‑case evidence: IOC lists, command/process trees, network traces.
KPI 7 — Ops & Collaboration Quality (SLA, Communication, Evidence)
What it means: SLA adherence (first response, escalation), report reproducibility, handoff quality, 24×7 coverage.
How to measure: SLA attainment %, mean first‑response time, case‑note quality checklist (IOCs, timeline, clear recommendations).
Benchmark: ≥95% SLA, <15 min first response, consistent evidence in every report.
Pro tip: Standardize case templates during PoC to make vendor comparison fair.
How to Run Your PoC (Quick Checklist)
Scenario set: ransomware initial access, credential dump, anomalous PowerShell, C2 beacon, O365 account takeover, exfil attempt.
Measure per scenario: MTTD, MTTR, Precision/Recall, FP count, response success, report quality (1–5).
Weighting example (100 pts): MTTD 20, MTTR 20, Precision/Recall 20, Noise 10, Coverage 15, Response 10, Ops 5.
Normalize time: dual timestamps; correct for ingestion delay.
Equal footing: same assets, same time window, same detections enabled.
Detect. Contain. Done. Breezeway’s got you covered.
Track 7 KPIs: MTTD, MTTR(-C), Precision/Recall, Noise Index, Coverage, Response Success Rate, and Ops & Collaboration Quality (SLA/reporting).
Benchmarks (typical enterprise targets):
MTTD: minutes—< 15–30 min for top-tier
MTTR(-C): < 60 min to contain/respond (shorter for high‑risk)
Precision/Recall: >80% / >90% in PoC scenarios
Noise Index: operator‑digestible; e.g., < 10 alerts/day per 1,000 EP
Coverage: >90% key assets onboarded; >80% priority TTP coverage
Response Success: >90% of high‑risk incidents contained/eradicated
Ops Quality: SLA ≥95%, <15 min first response; IOC + timeline + clear actions in every case note
KPI 1 — MTTD (Mean Time To Detect)
What it means: Time from threat occurrence to first MDR detection/flag.
How to measure: Difference between the attack simulation timestamp (MITRE ATT&CK‑based; e.g., ransomware initial access, credential dump) and the first detection alert time.
Benchmark: Minutes to tens of minutes; top‑tier < 15–30 min.
Pro tip: Record both UTC and local timestamps; correct for data ingestion latency.
KPI 2 — MTTR‑C / MTTR (Contain / Respond)
What it means: Time from detection to containment/blocking and actionable guidance completed.
How to measure: First detection alert → endpoint isolation / account disable / IOC block (or playbook action) completed.
Benchmark: < 60 min (tighter for high‑risk); automation shortens this materially.
Pro tip: Separate acknowledge, contain, and eradicate to see where time is spent.
KPI 3 — Detection Quality (Precision & Recall)
Precision = TP / (TP + FP): share of alerts that were real → reduces waste.
Recall = TP / (TP + FN): share of real threats caught → reduces misses.
How to measure: Pre‑agreed red‑team/ATT&CK emulation scenarios; label ground truth.
Benchmark: PoC goal Precision >80%, Recall >90% (tune by environment).
Pro tip: Track per‑scenario metrics (e.g., C2 beacon vs. LOLBins) to avoid averages hiding gaps.
KPI 4 — Noise Index (False Positives & Alert Load)
What it means: Volume and ratio of “unnecessary alerts” over day/week.
How to measure: FP/day, FP%, and analyst‑actionable alerts per 100 endpoints/day.
Benchmark: Operator‑digestible; e.g., < 10/day per 1,000 endpoints.
Pro tip: Require vendors to turn on deduplication and suppression they’d recommend in production.
KPI 5 — Coverage (Visibility & ATT&CK Technique Breadth)
What it means: Which telemetry/logs are collected (EPP/EDR, IDP, email, cloud, network) and which ATT&CK TTPs are detectable.
How to measure: % asset/platform onboarding + % ATT&CK T/TA covered + data latency.
Benchmark: >90% of critical assets onboarded; >80% of priority TTPs (ransomware, credential abuse, Living‑off‑the‑Land, data exfiltration).
Pro tip: List required connectors by “Required / Optional” and verify ingest health.
KPI 6 — Response Success Rate
What it means: Share of high‑risk incidents where risk was actually reduced (contain/isolate/block and/or customer‑executed recommendation).
How to measure: Count success vs. misses across high‑risk cases; if MDR is advisory‑only, include customer action rate.
Benchmark: >90% within PoC scope.
Pro tip: Ask for per‑case evidence: IOC lists, command/process trees, network traces.
KPI 7 — Ops & Collaboration Quality (SLA, Communication, Evidence)
What it means: SLA adherence (first response, escalation), report reproducibility, handoff quality, 24×7 coverage.
How to measure: SLA attainment %, mean first‑response time, case‑note quality checklist (IOCs, timeline, clear recommendations).
Benchmark: ≥95% SLA, <15 min first response, consistent evidence in every report.
Pro tip: Standardize case templates during PoC to make vendor comparison fair.
How to Run Your PoC (Quick Checklist)
Scenario set: ransomware initial access, credential dump, anomalous PowerShell, C2 beacon, O365 account takeover, exfil attempt.
Measure per scenario: MTTD, MTTR, Precision/Recall, FP count, response success, report quality (1–5).
Weighting example (100 pts): MTTD 20, MTTR 20, Precision/Recall 20, Noise 10, Coverage 15, Response 10, Ops 5.
Normalize time: dual timestamps; correct for ingestion delay.
Equal footing: same assets, same time window, same detections enabled.
Detect. Contain. Done. Breezeway’s got you covered.