Sampling checks Qevlar's verdicts against your own analysts. On a share of alerts, Qevlar investigates as usual but seals its result; an analyst investigates the alert without seeing it, and the two verdicts are then compared. Their alignment over a rolling 28 days tells you whether Qevlar's quality holds, including on the clusters you have automated.
This page covers how sampling works, how to activate it on a profile, what analysts see on a sealed alert, and how to read the results. For automating a cluster, see Automating response.
Why it matters
An independent measure. The alignment shown on the Automations page comes from feedback that analysts give after reading Qevlar's verdict. A sampled verdict is given before they see it, so the alignment it produces is Qevlar's unbiased accuracy on your alerts.
The control stays on after you automate. Alerts in an activated clusters keep being sampled, so a drift in Qevlar's verdicts shows up as a number moving on the Sampling page rather than going unnoticed.
Analysts keep their judgment. A sampled alert is a full manual investigation, and Qevlar's result only appears once the analyst has committed to a verdict. That verdict is the one that stands on the alert.
You decide the scope. Sampling is off by default. You activate it per profile, choose the share of alerts and how long a result stays sealed, and can keep chosen severities or VIP users out of the draw.
Where to find it
Platform UI. Open Sampling in the left navigation. Each profile shows as Ready to activate, Active or Paused, and an active profile has its own page with its verdict alignment and the list of misalignments. Analysts find sealed alerts under Investigations with the Sealed filter, and every sampled alert has a Sampling tab in its report.
API. A sealed alert returns SEALED in place of the verdict until the seal lifts. See Sampled alerts in the API below.
Before you start
You have the role to activate. Only administrators and managers can see the Sampling page and configure sampling. Analysts handle sealed alerts from Investigations.
Your SOAR handles SEALED. While an alert is sealed, the API returns SEALED in the verdict field. Route those alerts to manual review, as you do for suspicious ones, otherwise a sealed malicious alert falls through your workflow. If you only use the Qevlar interface, there is nothing to change.
Your analysts have the capacity. Every sampled alert is a full manual investigation. You chose the rate depending on the workload that is acceptable for your analysts.
There is no history before activation. Sampling measures from the day you activate it, and the first verdict alignment appears after 28 days, one full rolling window.
How sampling works
- Qevlar picks alerts at random as they arrive, cluster by cluster, at the share you set for the profile. Alerts that match one of your exclusions, or that already belong to a group, are never picked.
- Qevlar investigates the alert in full and seals the result. The analyst sees the alert as it arrived, with its raw events and a countdown. No verdict is returned and no automation runs while the alert is sealed.
- The analyst investigates, gives a verdict and explains it.
- Qevlar's result unseals and the two verdicts sit side by side. The analyst's verdict stands on the alert, and any automation on its cluster runs on that verdict.
If nobody gives a verdict before the sealing window ends, the alert unseals on its own, Qevlar's verdict applies, and the alert is marked Not counted. Verdict alignment is the share of sampled alerts with an analyst verdict where both verdicts match, over a rolling 28 days: each day's figure covers the 28 days ending that day, so a change in quality shows up without waiting for a period to end.
Activating sampling on a profile
Settings are per profile, so an MSSP can activate sampling for one client without touching the others.
- Open Sampling, pick a profile marked Ready to activate and click Activate.
- Choose the share of alerts to sample: 2%, 5%, 10% or a custom value. 5% is recommended, and the screen shows how many alerts a week that represents on this profile.
- If some alerts must never wait on a manual investigation, click Exclude some alerts and add a condition on severity or on VIP users. Excluded alerts are investigated and returned as usual and do not count toward alignment.
- Set the sealing window: 6, 8, 12 or 24 hours, with 8 hours recommended. A longer window gives more sampled alerts an analyst verdict, while a shorter one lets automation act sooner. Preview the sealed alert shows exactly what your analysts will see.
- Under Configure the API, check that your integration handles
SEALED. The API docs link opens the relevant section. - Click Activate and confirm. The confirmation sums up the workload and the sealing window, and sealing starts with the next alert ingested.
| Setting | Options | Recommended |
|---|---|---|
| Share of alerts sampled | 2%, 5%, 10% or a custom value | 5% |
| Sealing window | 6, 8, 12 or 24 hours | 8 hours |
| Exclusions | Severity or VIP users | None |
These settings can be changed later from the profile page, once sampling is paused. See Pausing, editing and resuming below.
What analysts see
Sealed alerts stay in the normal Investigations queue, marked Sealed in the Verdict column, and the Sealed filter lists them with a count. Analysts can tell an alert is sampled; only Qevlar's result is hidden.
- Open the alert. It reads Awaiting your verdict, Qevlar's result is sealed, with a countdown to the automatic unseal, the alert definition and the raw events.
- Investigate with your own tools, as you would without Qevlar.
- Under Investigate before it unseals, choose Benign, Suspicious or Malicious, explain your reasoning under Why? and click Share and Unseal.
Qevlar's result unseals and the report opens on its Sampling tab, which reads Verdict aligned or Misalignment and shows both reasonings. The analyst's verdict stays on the alert, and the full report is available in the other tabs. When the verdicts differ, the analyst's comment feeds Qevlar's feedback like any other correction and can become a context suggestion.
Reading the results
Open Sampling and select an active profile.
| Card | What it shows |
|---|---|
| Verdict alignment | The share of sampled verdicts that matched Qevlar's over the last 28 days, and how it has moved since activation |
| Alerts sampled | How many of the eligible alerts were sampled over the last 28 days, and roughly how many a week |
| Automations covered | How many of the profile's automations had at least one alert sampled in the last 28 days |
| Misalignments | Every sampled alert where the verdicts differed, with both verdicts and both reasonings |
For the first 28 days, Verdict alignment reads In progress with the date the first figure should be ready. Each profile is measured on its own, with no pooled figure across profiles. When alignment drops, start with Misalignments: sampling shows which alerts diverged and why the analyst disagreed, but it does not diagnose the cause, such as a missing connector.
Sampled alerts in the API
While an alert is sealed, GET /v2/investigations/{alert_id}/result returns SEALED in verdict and in investigation.report.outcome, and nothing from Qevlar's investigation: observables, report sections and severity come back empty or null. The message field says that analyst review is needed, and sealed_until gives the time at which the alert unseals on its own if nobody has given a verdict.
Route alerts whose verdict is SEALED to manual review, as you do for suspicious ones, and read the result again once the alert has unsealed. If nobody gives a verdict, the alert unseals within 15 minutes after sealed_until; from then on verdict carries the verdict that stands on the alert, with the rest of Qevlar's result.