Fixing the "Attempt to Bypass Conditional Access Rule" Analytic Rule
Alright class.
Fourth lesson in this series. We have fixed "MFA Rejected by User", "Privileged Role Assigned Outside PIM", and "New User Assigned to Privileged Role". Today we are taking apart "Attempt to bypass conditional access rule in Microsoft Entra ID" from the content hub.
Before anything else, a warning. The previous three rules had a sound core that needed better scoping and enrichment. This one is different. The core thesis is wrong, and no amount of tuning fixes a wrong thesis. So this post is less of a touch-up and more of a teardown; the detection logic underneath is rebuilt from scratch, because the original is detecting the wrong thing.
Failure Is Not Bypass
Here is the line the entire original rule is built on:
| where ConditionalAccessStatus == 1 or ConditionalAccessStatus =~ "failure"
A Conditional Access status of failure means the policy was evaluated and the user did not satisfy it. The control did its job and blocked the sign-in. That is the system working exactly as designed. Calling it an "attempt to bypass" is backwards. It is the opposite of a bypass.
Even in a healthy tenant, CA failures are one of the highest-volume events you have. A user on a non-compliant laptop, a user who has not done MFA yet on this session, a user signing in from a country a location policy excludes, a stale token, a half-finished registration. All of these produce a failure, and all of them are the policy correctly refusing access. Building an alert on top of "CA failure" means building an alert that fires constantly on the control succeeding.
A bypass is the inverse event. A bypass is when the control should have stopped the sign-in and did not. The user reached the resource without satisfying the policy. That is what we actually want to catch, and it does not look like a failure in the logs. It looks like a success.
So the whole rule needs to be turned around. Stop counting failures. Start finding the successes that should not have happened.
What a Bypass Actually Looks Like
Two shapes cover the overwhelming majority of real CA bypasses, and they are very different from each other.
The first is structural. Conditional Access grant controls, the MFA requirement, the compliant-device requirement, attach to modern authentication. Legacy and non-modern auth protocols, IMAP, POP3, SMTP, MAPI, Exchange ActiveSync, the bucket Entra labels "Other clients", largely predate the ability to interrupt a sign-in for MFA. An attacker with a valid username and password who authenticates over one of these protocols can walk straight past a grant control that would have stopped them in a browser. They do not fail the policy. The policy never gets the chance to apply. The fix Microsoft recommends is to block legacy auth outright, and if you have done that everywhere this branch will be quiet, which is exactly what you want. Most tenants have not finished that work, which is why this is still the single most reliable CA bypass signal in the wild. The interesting event here is a successful authentication over a legacy client.
The second is functional, and it is the closer match to the rule's actual name. An enabled, enforcing Conditional Access policy reports a per-policy result of failure on a sign-in that nonetheless succeeded. The gate said no, the user got in anyway. In a correctly behaving tenant this should be close to nonexistent, because a failed grant control should fail the sign-in. When it does happen it is worth a human look every time: a misconfigured policy, a fail-open session control, an exclusion that should not be there, or genuine evasion. This is low volume and high value, the opposite of the original rule.
The rewrite builds both shapes as separate branches and unions them, tagging each row with a Signal field so the analyst knows which one fired.
The Non-Interactive Logs Question
I've seen quite a few discussions about whether using AADNonInteractiveUserSignInLogs table is worth it.
let aadSignin = aadFunc("SigninLogs");
let aadNonInt = aadFunc("AADNonInteractiveUserSignInLogs");
union isfuzzy=true aadSignin, aadNonInt
The above part (from the original query) runs the identical failure-counting logic across both tables and unions them. That is the wrong way to use the non-interactive table, and it is wrong for the same reason the whole rule is wrong. AADNonInteractiveUserSignInLogs is the largest sign-in table in most tenants by a wide margin. It is token refreshes, background service auth, app-to-app traffic. Counting CA failures across all of that gives you volume, cost, and noise, and almost no signal.
But that does not mean the table is useless here, and this is the nuance. Legacy-protocol authentication shows up in the non-interactive table. So does a lot of password-based service auth. The structural-bypass branch, legacy auth that succeeded, is precisely the signal that lives there. So the answer is not "use both tables" and it is not "drop non-interactive". It is: use each table for the signal it is actually good at. The functional fail-open branch runs against SigninLogs only, because that is an interactive concept. The structural legacy branch runs against both tables, tightly scoped to the legacy client list, because that is where legacy auth surfaces. Scope: do not union blindly.
Dropping the Multiple-IPs Heuristic
The only real detection logic in the original, underneath the failure filter, is this:
| where IPAddressCount > threshold and StatusDetails !has "MFA successfully completed"
The thesis is that a user hitting a CA failure from more than one IP in a day is suspicious. It fails in both directions, the same way the role rule's contains "Admin" did. It is too noisy, because a mobile user on cellular and wifi, anyone on a VPN, and any office behind a NAT all produce multiple IPs trivially. And it is too narrow, because a focused attacker working a single account from one residential proxy never trips it.
Worse, look at the grouping key:
by UserPrincipalName, UserId, AppDisplayName, tostring(Browser), tostring(OS), City, State, Region, Type
Browser and OS come from the user-agent string, which the attacker controls completely. City, State and Region come from geo-IP, which is coarse and spoofable. An attacker rotating user agents fragments their own activity across grouping buckets and slides under the IP threshold without trying. Never key a detection on fields the attacker sets.
The rewrite drops the multiple-IPs heuristic entirely as a primary signal and demotes it to context. A GrindBaseline subquery, the same pattern as the InitiatorHistory baseline from the last post, counts each user's CA failures over the last day keyed on the stable UserId, not the user agent. When a bypass success then fires for that user, PrecededByRepeatedFailures lights up if they had been grinding against the control first. Fail a lot, then succeed, is a genuine bypass-attempt arc. Just succeeding from two IPs is not.
The Blind Spot You Should Know About
This rule catches a bypass at sign-in time, the runtime exploitation of a gap. It does not catch someone weakening or deleting the policy itself (luckily you should already have another analytic rule covering that). If an attacker with the right role disables a CA policy, adds an exclusion for their account, or flips it to report-only, that is a configuration change in AuditLogs, not a sign-in event, and it never touches this query. That is the detection that maps most directly to the MITRE sub-technique below, and it is a separate rule. Do not deploy this and assume your Conditional Access coverage is complete. It is a post of its own, and probably the next one.
MITRE Mappings for the Updated Rule
Tactics: Defense Evasion and Credential Access. Slipping a control is evasion; doing it with a valid credential is credential access.
T1556 Modify Authentication Process, sub-technique T1556.009 Conditional Access Policies. This is the family mapping. Note the precise definition of the sub-technique is about tampering with or disabling CA policies, which is the configuration-change companion detection mentioned above. This rule covers the runtime side of the same technique: reaching a resource without the control engaging.
T1078 Valid Accounts, sub-technique T1078.004 Cloud Accounts. The legacy-auth branch is a valid cloud credential authenticating over a path the policy cannot reach.
T1110 Brute Force. Only relevant to the grind context, when a bypass success is preceded by a run of failures from the same account.
Rule Settings
Run every 60 minutes with a query period of 60 minutes. The original ran once a day, which for a successful bypass is far too slow. A daily cadence means an attacker can be inside for the better part of a day before the rule even looks. The grind baseline still reaches back a full day independently of the detection window. Medium severity, raised by the indicators. Alert per result. Group by Account entity with a 5 hour lookback window.
Entity mapping:
- Name and UPNSuffix to Account
- IPAddress to IP
Custom details to surface in the incident: Signal, RiskIndicators, ClientAppUsed, AppDisplayName, FailedPolicy, FailedPolicyResult, AuthenticationRequirement, RiskLevelDuringSignIn, PriorCAFailures.
KQL
// =====================================================================
// Conditional Access Bypass - True Fail-Open or Anomalous Legacy Auth
// Version : 2.1 | union type-collision fix
// =====================================================================
let DetectionWindow = 1h;
let BaselineWindow = 14d;
let BaselineCutoff = 2d;
let MinBaselineHits = 3;
let FireThreshold = 50;
let GrindThreshold = 5;
let LegacyAuthClients = dynamic(["Other clients","IMAP4","POP3","SMTP","Authenticated SMTP",
"MAPI","Exchange ActiveSync","Exchange Web Services",
"Exchange Online PowerShell","AutoDiscover"]);
let HighRiskLegacy = dynamic(["Other clients","IMAP4","POP3","Exchange Online PowerShell"]);
let BenignErrorCodes = dynamic(["50140"]);
let KnownServiceUPNs = dynamic([]);
let W_FailOpen = 60;
let W_NewClient = 40;
let W_NewNetwork = 25;
let W_HighRiskClient = 15;
let W_SignInRisk = 30;
let W_SingleFactor = 15;
let W_Privileged = 25;
let W_Grind = 15;
let W_ServiceAcct = -40;
let W_AppPassword = -25;
// --- Leg 1: interactive. Flatten to scalars (inside) ---
let LegacyInteractive =
SigninLogs
| where TimeGenerated > ago(BaselineWindow)
| where ResultType == "0"
| where ClientAppUsed in~ (LegacyAuthClients)
| extend DevJson = todynamic(DeviceDetail),
LocJson = todynamic(LocationDetails),
StsJson = todynamic(Status)
| project TimeGenerated, UserId, UserPrincipalName, ClientAppUsed, AppDisplayName,
ResourceDisplayName, IPAddress, ResultDescription, ConditionalAccessStatus,
AuthenticationRequirement, RiskLevelDuringSignIn, RiskState, CorrelationId, Id,
OS = tostring(DevJson.operatingSystem),
Browser = tostring(DevJson.browser),
DeviceId = tostring(DevJson.deviceId),
IsCompliant = tostring(DevJson.isCompliant),
City = tostring(LocJson.city),
Country = tostring(LocJson.countryOrRegion),
StatusErrorCode = tostring(StsJson.errorCode),
StatusDetails = tostring(StsJson.additionalDetails),
SourceTable = "SigninLogs";
// ---2: non-interactive
let LegacyNonInteractive =
AADNonInteractiveUserSignInLogs
| where TimeGenerated > ago(BaselineWindow)
| where ResultType == "0"
| where ClientAppUsed in~ (LegacyAuthClients)
| extend DevJson = todynamic(DeviceDetail),
LocJson = todynamic(LocationDetails),
StsJson = todynamic(Status)
| project TimeGenerated, UserId, UserPrincipalName, ClientAppUsed, AppDisplayName,
ResourceDisplayName, IPAddress, ResultDescription, ConditionalAccessStatus,
AuthenticationRequirement, RiskLevelDuringSignIn, RiskState, CorrelationId, Id,
OS = tostring(DevJson.operatingSystem),
Browser = tostring(DevJson.browser),
DeviceId = tostring(DevJson.deviceId),
IsCompliant = tostring(DevJson.isCompliant),
City = tostring(LocJson.city),
Country = tostring(LocJson.countryOrRegion),
StatusErrorCode = tostring(StsJson.errorCode),
StatusDetails = tostring(StsJson.additionalDetails),
SourceTable = "AADNonInteractiveUserSignInLogs";
let LegacySlice = materialize(
union isfuzzy=true LegacyInteractive, LegacyNonInteractive
| where UserPrincipalName !in~ (KnownServiceUPNs)
| extend NetworkPrefix = iff(IPAddress contains ":",
strcat(strcat_array(array_slice(split(IPAddress, ":"), 0, 2), ":"), "::"),
strcat_array(array_slice(split(IPAddress, "."), 0, 2), "."))
);
let ClientBaseline = LegacySlice
| where TimeGenerated between (ago(BaselineWindow) .. ago(BaselineCutoff))
| summarize Hits = count() by UserId, ClientAppUsed
| where Hits >= MinBaselineHits
| project UserId, ClientAppUsed;
let NetworkBaseline = LegacySlice
| where TimeGenerated between (ago(BaselineWindow) .. ago(BaselineCutoff))
| summarize Hits = count() by UserId, ClientAppUsed, NetworkPrefix
| where Hits >= MinBaselineHits
| project UserId, ClientAppUsed, NetworkPrefix;
let RecentLegacy = LegacySlice | where TimeGenerated > ago(DetectionWindow);
// --- Branch A: legacy protocol this identity has never used ---
let NewClientHits = RecentLegacy
| join kind=leftanti ClientBaseline on UserId, ClientAppUsed
| extend Signal = "NewLegacyClient", FailedPolicy = "";
// --- Branch B: established legacy user, high-risk protocol, unseen infrastructure
let NewNetworkHits = RecentLegacy
| join kind=leftanti NetworkBaseline on UserId, ClientAppUsed, NetworkPrefix
| join kind=leftsemi ClientBaseline on UserId, ClientAppUsed
| where ClientAppUsed in~ (HighRiskLegacy)
| extend Signal = "LegacyFromNewNet", FailedPolicy = "";
// --- Branch C: true fail-open. SigninLogs only
let FailOpenHits =
SigninLogs
| where TimeGenerated > ago(DetectionWindow)
| where ResultType == "0"
| where ConditionalAccessStatus =~ "failure"
| extend DevJson = todynamic(DeviceDetail),
LocJson = todynamic(LocationDetails),
StsJson = todynamic(Status)
| mv-apply CAP = todynamic(ConditionalAccessPolicies) on (
where tostring(CAP.result) =~ "failure"
and isnotempty(CAP.enforcedGrantControls)
and tostring(CAP.enforcedGrantControls) !has "Block"
| project FailedPolicy = tostring(CAP.displayName)
)
| project TimeGenerated, UserId, UserPrincipalName, ClientAppUsed, AppDisplayName,
ResourceDisplayName, IPAddress, ResultDescription, ConditionalAccessStatus,
AuthenticationRequirement, RiskLevelDuringSignIn, RiskState, CorrelationId, Id,
OS = tostring(DevJson.operatingSystem),
Browser = tostring(DevJson.browser),
DeviceId = tostring(DevJson.deviceId),
IsCompliant = tostring(DevJson.isCompliant),
City = tostring(LocJson.city),
Country = tostring(LocJson.countryOrRegion),
StatusErrorCode = tostring(StsJson.errorCode),
StatusDetails = tostring(StsJson.additionalDetails),
SourceTable = "SigninLogs",
NetworkPrefix = "",
FailedPolicy,
Signal = "CAFailOpen";
// --- Context ---
let GrindBaseline = materialize(
SigninLogs
| where TimeGenerated > ago(1d)
| where ConditionalAccessStatus =~ "failure"
| summarize PriorCAFailures = count(), DistinctFailIPs = dcount(IPAddress) by UserId
);
let IdentityContext = materialize(
IdentityInfo
| where TimeGenerated > ago(BaselineWindow)
| summarize arg_max(TimeGenerated, AccountUPN, AccountCreationTime, Department, JobTitle,
Manager, AssignedRoles, IsMFARegistered, IsAccountEnabled,
IsServiceAccount, UserType) by AccountUPN
| extend UPNKey = tolower(AccountUPN),
PrivilegedTarget = array_length(todynamic(AssignedRoles)) > 0
| project UPNKey, AccountCreationTime, Department, JobTitle, Manager, AssignedRoles,
IsMFARegistered, IsAccountEnabled, IsServiceAccount, UserType, PrivilegedTarget
);
union isfuzzy=true NewClientHits, NewNetworkHits, FailOpenHits
| where StatusErrorCode !in (BenignErrorCodes)
| extend UPNKey = tolower(UserPrincipalName)
| lookup kind=leftouter (IdentityContext) on UPNKey
| lookup kind=leftouter (GrindBaseline) on UserId
| extend PriorCAFailures = coalesce(PriorCAFailures, 0),
DistinctFailIPs = coalesce(DistinctFailIPs, 0)
| extend W1 = iff(Signal == "CAFailOpen", W_FailOpen, 0),
W2 = iff(Signal == "NewLegacyClient", W_NewClient, 0),
W3 = iff(Signal == "LegacyFromNewNet", W_NewNetwork, 0),
W4 = iff(ClientAppUsed in~ (HighRiskLegacy), W_HighRiskClient, 0),
W5 = iff(tolower(RiskLevelDuringSignIn) in ("high", "medium"), W_SignInRisk, 0),
W6 = iff(AuthenticationRequirement =~ "singleFactorAuthentication", W_SingleFactor, 0),
W7 = iff(PrivilegedTarget == true, W_Privileged, 0),
W8 = iff(PriorCAFailures >= GrindThreshold, W_Grind, 0),
W9 = iff(IsServiceAccount == true, W_ServiceAcct, 0),
W10 = iff(StatusDetails has "app password", W_AppPassword, 0)
| summarize
FirstSeen = min(TimeGenerated),
TimeGenerated = max(TimeGenerated),
EventCount = count(),
SampleIP = take_any(IPAddress),
IPAddresses = make_set(IPAddress, 10),
Countries = make_set(Country, 5),
Cities = make_set(City, 5),
Apps = make_set(AppDisplayName, 10),
Resources = make_set(ResourceDisplayName, 10),
Policies = make_set_if(FailedPolicy, isnotempty(FailedPolicy), 5),
Devices = make_set_if(DeviceId, isnotempty(DeviceId), 5),
StatusDetail = take_any(StatusDetails),
RolesHeld = take_any(AssignedRoles),
CorrelationId = take_any(CorrelationId),
SourceTables = make_set(SourceTable, 3),
W1 = max(W1), W2 = max(W2), W3 = max(W3), W4 = max(W4), W5 = max(W5),
W6 = max(W6), W7 = max(W7), W8 = max(W8), W9 = min(W9), W10 = min(W10)
by UserPrincipalName, UserId, Signal, ClientAppUsed, ConditionalAccessStatus,
AuthenticationRequirement, Department, JobTitle, Manager, IsMFARegistered,
IsAccountEnabled, IsServiceAccount, UserType, PrivilegedTarget,
AccountCreationTime, PriorCAFailures, DistinctFailIPs
| extend RiskScore = W1 + W2 + W3 + W4 + W5 + W6 + W7 + W8 + W9 + W10
| where RiskScore >= FireThreshold
| extend RiskIndicators = trim(@"\s\|\s*$", strcat(
iff(W1 != 0, "CAControlFailedButAllowed | ", ""),
iff(W2 != 0, "FirstEverLegacyClient | ", ""),
iff(W3 != 0, "LegacyFromNewNetwork | ", ""),
iff(W4 != 0, "HighRiskLegacyProtocol | ", ""),
iff(W5 != 0, "ElevatedSignInRisk | ", ""),
iff(W6 != 0, "SingleFactorOnly | ", ""),
iff(W7 != 0, "PrivilegedTarget | ", ""),
iff(W8 != 0, "PrecededByFailures | ", ""),
iff(W9 != 0, "SuppressServiceAcct | ", ""),
iff(W10 != 0, "SuppressAppPassword | ", "")
))
| extend TargetAccountAgeDays = datetime_diff('day', now(), AccountCreationTime),
Name = tostring(split(UserPrincipalName, "@")[0]),
UPNSuffix = tostring(split(UserPrincipalName, "@")[1])
| project
TimeGenerated, FirstSeen, Signal, RiskScore, RiskIndicators, EventCount,
UserPrincipalName, Name, UPNSuffix, UserId,
ClientAppUsed, Apps, Resources, Policies,
ConditionalAccessStatus, AuthenticationRequirement,
IPAddress = SampleIP, IPAddresses, Cities, Countries, Devices, StatusDetail,
PriorCAFailures, DistinctFailIPs,
Department, JobTitle, Manager, RolesHeld, PrivilegedTarget,
IsMFARegistered, IsAccountEnabled, IsServiceAccount, UserType,
TargetAccountAgeDays, CorrelationId, SourceTables
| sort by RiskScore desc, TimeGenerated descYou can also download this as an analytic rule and import it directly to Sentinel.
Follow my repo - GitHub
What You Should Do Next
- Trim the LegacyAuthClients list to reality. List the legacy client values that still appear in your tenant and have not been blocked. If you have already blocked legacy auth everywhere, this branch will be silent, and that silence is the goal, not a bug. If it is loud, you have an exposure to close, not just an alert to triage.
- Run the query manually over the last 7 days before deploying, one branch at a time. The LegacyAuthSuccess branch tells you your real legacy-auth footprint. The FailOpenControl branch should return very little; if it returns a lot, you have a CA misconfiguration to investigate before this is a detection at all.
- Decide whether you want the SensitiveApps list populated. Left empty, the rule watches every app, which is the right default to start. Once you know your baseline, narrowing SingleFactorOnly and SensitiveAppAccessed to the apps that actually matter sharpens the severity signal.
- Build the companion detection for Conditional Access policy changes in AuditLogs (or use the built in from Microsoft) This rule catches the bypass at sign-in; it does not catch someone disabling a policy, adding an exclusion, or flipping it to report-only. That is the T1556.009 configuration-change side, and it is the next post.
Class dismissed