Fixing the "Mass Secret Retrieval from Azure Key Vault" Analytic Rule
Alright class.
This time let's take a trip outside Entra into the data plane of an Azure resource. The rule is "Mass secret retrieval from Azure Key Vault", and the premise is exactly right: a single caller pulling secrets at scale is either a credential dump in progress or an application misbehaving, and both deserve eyes. Then you read which operations it counts, and how it counts them.
Two of the Three Operations Cannot Return a Secret
The rule counts three operations as retrieval:
let OperationList = dynamic(["SecretGet", "KeyGet", "VaultGet"]);
VaultGet reads the properties of the vault object, tags, SKU, access configuration, through the management plane. It cannot return a secret. KeyGet returns the public part of a stored key; the private material never leaves the vault, that is the entire point of the product. The only operation in that list that hands over secret material is SecretGet, and it also carries certificate private keys, because downloading a certificate with its key goes through SecretGet on the certificate's backing secret.
The rule already knows VaultGet is a problem, because it ships with an allowlist of two Microsoft service AppIds that hammer VaultGet all day for indexing and policy checks. When an operation needs a service allowlist to stay quiet, that is the operation telling you it does not belong in the detection. Count only what can steal, and the allowlist dies with the operation it existed to silence.
Thresholds That Fight Each Other
Two gates decide what fires: more than ten distinct request URIs, and more than fifty events. Walk an actual dump through them. An attacker lists the vault and reads thirty secrets, once each, because nobody reads a stolen secret twice. Thirty distinct URIs, well past ten. Thirty events, under fifty. Dropped. The distinct gate says mass, the event gate throws it away, and the description advertises the event count as the knob to tune.
The distinct count is broken in the other direction too. Request URIs include secret versions, so rotating one secret eleven times and reading each version back is eleven distinct URIs, one secret, and congratulations, you are a mass retrieval incident. And the description's answer to all of this is that you should tweak the thresholds to your environment's average and filter known sources by hand from historical analysis. That is the rule outsourcing its baseline to you, permanently.
What a Dump Actually Looks Like
Microsoft's own incident response team has written up the choreography: list, then get. SecretList to learn the names, SecretGet to take the values, CertificateList then SecretGet for certificates with their private keys. Through the portal the pattern is the same with VaultGet in front. The enumeration is the tell, because legitimate consumers know the name of the secret they came for.
Then ask who does this. A stolen token or service principal credential, dumping from infrastructure the real owner never uses. An insider pulling everything before resignation, from their usual desk, inside their usual hours. A brand-new identity whose first recorded action anywhere on the Key Vault data plane is a bulk read. Three different actors, one shared property: the volume is abnormal for that caller, even when it would be normal for someone else in the tenant.
That is the baseline the rule should have had. Not a flat number tuned by hand, but each caller's own history: their best day of distinct reads, their known addresses, their known vaults. The attacker cannot cheaply join that baseline, because the only way in is fourteen days of looking like the caller they stole.
The Backup Job Is Not an Incident
The benign process that owns this telemetry is automation. Backup jobs, deployment pipelines and secret-sync tooling read dozens of distinct secrets on schedule, every day, and any flat threshold either alerts on them daily or is raised until it sees nothing at all. Under a self-baseline they carry their own alibi: yesterday looked like today, so the anomaly signal never fires, the IPs match history, the vaults match history, and the caller scores zero. No AppId allowlist, no hand-maintained exclusions, and rotation tooling stays quiet because versions collapse to one secret name. On a normal day this rule should return nothing.
What does fire, fires for a reason. A caller at three times its own best day fires alone. An identity with no Key Vault history at all doing a bulk read fires alone. Enumeration, new addresses, new vaults and denied-read bursts stack on top.
The Rebuild
Two gates, in order:
| where DistinctSecrets >= MassFloor
...
| where RiskScore >= FireThreshold
Every signal reads from AzureDiagnostics; the identity and service principal tables only decorate. New-IP and new-vault checks evaluate only for callers that have history, so a brand-new caller cannot double-fire on everything being new, and a missing lookup silences a signal rather than inventing one.
What the Analyst Sees
The row is the investigation. DistinctSecrets next to BaselineDailyMax shows the anomaly as two numbers. NewIps and NewVaults list exactly what changed about the caller's behaviour. SecretsSample names what was read, which on a true positive is the rotation list, ready to hand to the platform team. Humans arrive with roles, department and user type; apps arrive with their display name resolved from the service principal sign-in logs instead of a bare AppId. Nobody has to open Logs to know what happened.
The RiskIndicators Field
Same accumulating string as the rest of the series, mapped as a custom detail:
VolumeAnomaly | NewCallerIdentity | SecretEnumeration | NewSourceIp | NewVaultAccess | FailureBurst | OffHoursActivity
The combination to chase is VolumeAnomaly with NewSourceIp: a known identity reading far past its own record from an address it has never used is the stolen-credential shape end to end. NewCallerIdentity with anything on top is a burner.
The Blind Spots You Should Know About
An insider whose norm is already reading everything lives inside their own baseline; behavioural rules cannot catch someone who has always behaved this way. And the targeted theft of one high-value secret never crosses the mass floor; that is a crown-jewel watchlist rule, not this one.
In addition to that, my hunting queries should help you in further investigation around key vaults
Bartosz Wysocki
MITRE Mappings for the Updated Rule
Tactic: Credential Access.
T1555.006 Credentials from Password Stores, Cloud Secrets Management Stores. The technique is this rule verbatim: harvesting credentials from a cloud secrets manager.
Rule Settings
Run every 60 minutes, and set the query period to 14 days so the per-caller baselines resolve. High severity, because a firing row means a caller tripled its own history or appeared from nowhere reading at scale. Alert per result, since the query collapses to one row per caller. Group by the Account entity over 8 hours so a dump spanning several runs becomes one incident.
Entity mapping:
Name to Account (Name), UPNSuffix to Account (UPNSuffix)
PrimaryIp to IP (Address)
VaultResourceId to AzureResource (ResourceId)
Custom details: RiskScore, RiskIndicators, DistinctSecrets, BaselineDailyMax, GetCount, ListCount, BackupCount, FailCount, SecretsSample, VaultsAccessed, NewIps, NewVaults, SpName, CallerAppId, CallerRoles, Department, UserType, CallerType, UserAgent.
KQL
// =====================================================================
// Key Vault Mass Secret Retrieval - Behavioural
// =====================================================================
// Description : Detects a caller reading an unusual number of distinct Key Vault secrets,
// judged against that caller's own history rather than a flat threshold. Counts
// only operations that return secret material (SecretGet and the backup
// operations), normalises request URIs to secret names so version churn cannot
// inflate the count, and scores enumeration, new source IPs, new vaults, a
// brand-new caller identity and failure bursts. Callers are named via managed
// identity and service principal sign-ins plus a catalogue of Microsoft
// first-party AppIds, which also damps the new-caller signal for first-seen
// Microsoft services; human callers are never damped.
// Type : Detection
//
// Tables : AzureDiagnostics, IdentityInfo, AADServicePrincipalSignInLogs,
// AADManagedIdentitySignInLogs, externaldata (Microsoft first-party app catalogue)
// Connectors : Azure Key Vault diagnostic settings (AuditEvent to Log Analytics),
// Microsoft Entra ID (ServicePrincipal and ManagedIdentity sign-in logs),
// Microsoft Sentinel UEBA (IdentityInfo)
// License : Microsoft Sentinel; Key Vault diagnostics carry Azure Monitor ingestion cost,
// UEBA and the sign-in tables only enrich; every signal reads from AzureDiagnostics
//
// Tuning : - Set the rule query period to P14D; the 14d baselines only resolve if the rule looks back that far
// - MassFloor - distinct secrets in one window before a caller is even considered
// - AnomalyMultiplier - how far past its own best day a caller must go to be anomalous
// - FailureFloor - denied secret reads in the window that count as a probe
// - BusinessStart / BusinessEnd - local working hours; TimeGenerated is UTC, shift to your tenant
// - The Microsoft app catalogue loads over externaldata from GitHub; mirror it into a
// Sentinel watchlist if a detection with an internet dependency bothers you (it should)
// - Widen DetectionWindow past the run frequency if slow dumps split across runs
//
// Known FPs : - A pipeline or backup job after an infrastructure move fires once on NewSourceIp;
// confirm the change and close, the new address joins its history
// - A genuinely new deployment identity doing its first big pull fires on
// NewCallerIdentity; one look at a brand-new mass reader is the point
// - A newly enabled Microsoft service reading vaults stays quiet; the catalogue damps
// the new-caller signal for service identities only
//
// Author : Bartosz Wysocki | https://www.itprofessor.cloud
// Version : 1.0 | 2026-07-04
// =====================================================================
let DetectionWindow = 1h;
let BaselineLookback = 14d; // per-caller history: daily norms, known IPs, known vaults
let IdentityLookback = 14d; // UEBA context for human callers
let SpLookback = 14d; // service principal and managed identity sign-ins, to name app callers
let MassFloor = 10; // distinct secrets in the window before a caller is considered
let AnomalyMultiplier = 3; // multiple of the caller's own best day that counts as anomalous
let FailureFloor = 10; // denied secret reads that count as permission probing
let BusinessStart = 7;
let BusinessEnd = 19;
let FireThreshold = 3;
// Scoring weights - a caller far beyond its own history, or with no history at all, fires alone
let W_VolumeAnomaly = 3; // distinct secrets at least AnomalyMultiplier times the caller's best day
let W_NewCaller = 3; // identity never seen on the Key Vault data plane in the lookback
let W_Enumeration = 2; // list operations alongside the reads; the dump choreography
let W_NewSourceIp = 2; // an address this caller has never used before
let W_NewVault = 2; // a vault this caller has never touched before
let W_FailureBurst = 2; // denied secret reads at volume; probing for reach
let W_OffHours = 1; // activity outside working hours
let ValueOps = dynamic(["SecretGet"]);
let EnumOps = dynamic(["SecretList", "SecretListVersions", "CertificateList", "KeyList"]);
// Microsoft first-party application catalogue (invictus-ir), used to NAME app callers and to damp
// exactly one signal (NewCallerIdentity) for service identities. Never used to exclude rows:
// Azure CLI, Azure PowerShell and the portal are all first-party AppIds, and excluding them
// excludes every human-driven retrieval
let MicrosoftApps = externaldata(id: string, displayName: string, appId: string, applicationType: string)
[@"https://raw.githubusercontent.com/invictus-ir/entra-apps/main/applist.csv"]
with (format = "csv", ignoreFirstRecord = true)
| project MsAppName = displayName, MsAppId = appId;
let MsAppIds = toscalar(MicrosoftApps | summarize make_set(MsAppId, 5000));
// All Key Vault data-plane audit rows in the lookback, with the caller resolved once.
// Materialised because both the detection window and the history read from it.
let KvData = materialize(
AzureDiagnostics
| where TimeGenerated > ago(BaselineLookback)
| where ResourceType =~ "VAULTS"
| where OperationName != "Authentication" // SDK pre-auth 401 handshake noise, carries no identity
| extend ResultType = column_ifexists("ResultType", ""),
requestUri_s = column_ifexists("requestUri_s", ""),
clientInfo_s = column_ifexists("clientInfo_s", ""),
CallerIPAddress = column_ifexists("CallerIPAddress", ""),
identity_claim_oid_g = column_ifexists("identity_claim_oid_g", ""),
identity_claim_upn_s = column_ifexists("identity_claim_upn_s", ""),
identity_claim_appid_g = column_ifexists("identity_claim_appid_g", ""),
claim_oid_long = column_ifexists("identity_claim_http_schemas_microsoft_com_identity_claims_objectidentifier_g", ""),
claim_upn_long = column_ifexists("identity_claim_http_schemas_xmlsoap_org_ws_2005_05_identity_claims_upn_s", "")
| extend CallerOid = iff(isempty(identity_claim_oid_g), claim_oid_long, identity_claim_oid_g)
| extend CallerUPN = tolower(iff(isempty(identity_claim_upn_s), claim_upn_long, identity_claim_upn_s))
| extend CallerAppId = identity_claim_appid_g
| extend Caller = case(isnotempty(CallerUPN), CallerUPN, isnotempty(CallerAppId), CallerAppId, CallerOid)
| where isnotempty(Caller)
| extend CallerType = iff(isnotempty(CallerUPN), "User", "App")
// secret name, version stripped, so rotation churn is one object, not many
| extend ObjectName = extract(@"/(secrets|keys|certificates)/([^/?]+)", 2, tolower(requestUri_s))
| extend IsValueOp = OperationName in (ValueOps) or OperationName endswith "Backup"
| extend IsEnumOp = OperationName in (EnumOps)
| extend HourOfDay = datetime_part("Hour", TimeGenerated)
| extend OffHoursEvent = HourOfDay < BusinessStart or HourOfDay >= BusinessEnd or dayofweek(TimeGenerated) in (0d, 6d)
);
let WindowEvents = KvData | where TimeGenerated > ago(DetectionWindow);
let History = KvData | where TimeGenerated <= ago(DetectionWindow);
// The caller's own norm: best single day of distinct secret reads in the history
let HistoryDaily = History
| where IsValueOp and ResultType =~ "Success" and isnotempty(ObjectName)
| summarize DailyDistinct = dcount(ObjectName) by Caller, Day = bin(TimeGenerated, 1d)
| summarize BaselineDailyMax = max(DailyDistinct) by Caller;
// The caller's known ground: addresses and vaults it has used before
let HistorySets = History
| summarize HistIpSet = make_set(CallerIPAddress, 500), HistVaultSet = make_set(Resource, 200) by Caller;
// Human caller context
let IdentityContext = IdentityInfo
| where TimeGenerated > ago(IdentityLookback)
| summarize arg_max(TimeGenerated, AssignedRoles, Department, UserType) by AccountUPN = tolower(AccountUPN);
// Display names for app callers: service principals by AppId, managed identities by object id
let SpNames = AADServicePrincipalSignInLogs
| where TimeGenerated > ago(SpLookback)
| summarize SpName = take_any(ServicePrincipalName) by AppId;
let MiNames = AADManagedIdentitySignInLogs
| where TimeGenerated > ago(SpLookback)
| where isnotempty(ServicePrincipalName) and isnotempty(ServicePrincipalId)
| summarize MiName = take_any(ServicePrincipalName) by ServicePrincipalId;
WindowEvents
| summarize
StartTime = min(TimeGenerated),
EndTime = max(TimeGenerated),
DistinctSecrets = dcountif(ObjectName, IsValueOp and ResultType =~ "Success" and isnotempty(ObjectName)),
GetCount = countif(IsValueOp and ResultType =~ "Success"),
ListCount = countif(IsEnumOp and ResultType =~ "Success"),
BackupCount = countif(OperationName endswith "Backup" and ResultType =~ "Success"),
FailCount = countif(IsValueOp and ResultType !~ "Success"),
SecretsSample = make_set_if(ObjectName, IsValueOp and isnotempty(ObjectName), 20),
VaultSet = make_set(Resource, 20),
IpSet = make_set(CallerIPAddress, 20),
PrimaryIp = take_any(CallerIPAddress),
VaultResourceId = take_any(ResourceId),
UserAgent = take_any(clientInfo_s),
CallerOid = take_any(CallerOid),
OffHoursCount = countif(OffHoursEvent)
by Caller, CallerType, CallerAppId
| where DistinctSecrets >= MassFloor
| join kind=leftouter HistoryDaily on Caller
| join kind=leftouter HistorySets on Caller
| join kind=leftouter IdentityContext on $left.Caller == $right.AccountUPN
| join kind=leftouter SpNames on $left.CallerAppId == $right.AppId
| join kind=leftouter MiNames on $left.CallerOid == $right.ServicePrincipalId
| join kind=leftouter MicrosoftApps on $left.CallerAppId == $right.MsAppId
| extend HasHistory = isnotnull(BaselineDailyMax)
| extend HistIpSet = iff(isnull(HistIpSet), dynamic([]), HistIpSet)
| extend HistVaultSet = iff(isnull(HistVaultSet), dynamic([]), HistVaultSet)
| extend NewIps = set_difference(IpSet, HistIpSet)
| extend NewVaults = set_difference(VaultSet, HistVaultSet)
| extend CallerRoles = tostring(AssignedRoles)
| extend IsMicrosoftApp = CallerAppId in (MsAppIds)
| extend CallerName = iff(CallerType == "User", Caller, coalesce(SpName, MiName, MsAppName, CallerAppId))
| extend sVolumeAnomaly = HasHistory and DistinctSecrets >= BaselineDailyMax * AnomalyMultiplier
// first-seen Microsoft SERVICE identities are expected platform behaviour; humans are never damped
| extend sNewCaller = not(HasHistory) and not(IsMicrosoftApp and CallerType == "App")
| extend sEnumeration = ListCount > 0
| extend sNewSourceIp = HasHistory and array_length(NewIps) > 0
| extend sNewVault = HasHistory and array_length(NewVaults) > 0
| extend sFailureBurst = FailCount >= FailureFloor
| extend sOffHours = OffHoursCount > 0
| extend RiskScore =
iff(sVolumeAnomaly, W_VolumeAnomaly, 0)
+ iff(sNewCaller, W_NewCaller, 0)
+ iff(sEnumeration, W_Enumeration, 0)
+ iff(sNewSourceIp, W_NewSourceIp, 0)
+ iff(sNewVault, W_NewVault, 0)
+ iff(sFailureBurst, W_FailureBurst, 0)
+ iff(sOffHours, W_OffHours, 0)
| where RiskScore >= FireThreshold
| extend Name = tostring(split(Caller, "@", 0)[0]), UPNSuffix = tostring(split(Caller, "@", 1)[0])
| extend Risk_1 = iff(sVolumeAnomaly, "VolumeAnomaly", "")
| extend Risk_2 = iff(sNewCaller, "NewCallerIdentity", "")
| extend Risk_3 = iff(sEnumeration, "SecretEnumeration", "")
| extend Risk_4 = iff(sNewSourceIp, "NewSourceIp", "")
| extend Risk_5 = iff(sNewVault, "NewVaultAccess", "")
| extend Risk_6 = iff(sFailureBurst, "FailureBurst", "")
| extend Risk_7 = iff(sOffHours, "OffHoursActivity", "")
| extend RiskIndicators = trim(@"\s\|\s*$", strcat(
iff(isnotempty(Risk_1), strcat(Risk_1, " | "), ""),
iff(isnotempty(Risk_2), strcat(Risk_2, " | "), ""),
iff(isnotempty(Risk_3), strcat(Risk_3, " | "), ""),
iff(isnotempty(Risk_4), strcat(Risk_4, " | "), ""),
iff(isnotempty(Risk_5), strcat(Risk_5, " | "), ""),
iff(isnotempty(Risk_6), strcat(Risk_6, " | "), ""),
iff(isnotempty(Risk_7), strcat(Risk_7, " | "), "")
))
| project
StartTime, EndTime, RiskScore, RiskIndicators,
Caller, CallerName, Name, UPNSuffix, CallerType, CallerAppId, IsMicrosoftApp,
PrimaryIp, IpSet, NewIps, VaultSet, NewVaults, VaultResourceId,
DistinctSecrets, GetCount, ListCount, BackupCount, FailCount, BaselineDailyMax,
SecretsSample, UserAgent, CallerRoles, Department, UserType, OffHoursCount
| sort by RiskScore desc, EndTime desc
You can also download this as an analytic rule and import it directly to Sentinel.
Follow my repo - GitHub
What You Should Do Next
Check your diagnostic settings first, because they decide whether this rule sees anything. Every vault needs AuditEvent flowing to the workspace, and this rule reads the AzureDiagnostics destination mode; a vault logging to resource-specific tables is a blind spot until you point a variant at those tables.
Run it over the last day next to the original and compare. The original returns whatever its flat thresholds happen to catch; this should return nothing on a quiet day, and each row it does return should read as a finished triage.
Set MassFloor to your environment. Ten distinct secrets in an hour is conservative for most tenants; if your busiest legitimate caller reads forty, the floor matters less than the baseline, which is the point.
Build the crown-jewel companion. A watchlist of your break-glass and domain-critical secret names, alerting on any read by anyone, covers the targeted theft this rule deliberately leaves alone.
Class dismissed.