Alright class.

Today it is "Successful logon from IP and failure from a different IP". The shape is this: a user signs in successfully to an app from one IP, and within ten minutes a sign-in to the same app fails from a different IP. The reading is that someone is guessing at an account that is in active legitimate use. That is a fair thing to look for. The rule then leans on two things to make it usable, a crude IP comparison and a UEBA score cutoff, and both are doing the wrong job. The result is noise held back by a single black-box filter.

All Failures Are Not Guessing

The biggest problem is the failure filter. The original takes every failure except a couple of interrupt codes: ResultType !in ("0", "50140"). That bucket is enormous and almost none of it is password guessing.

A Conditional Access block is a failure. An MFA prompt that times out is a failure. An expired password is a failure. A blocked legacy-auth attempt is a failure. The rule's own description admits it, that failed logons from known ranges are often benign depending on Conditional Access. So the rule fires on the user's own second device getting blocked by policy just as readily as on an attacker, and then relies on the UEBA cutoff to clean up the mess.

The threat model is password guessing, so the failure has to be a credential failure. Three result types mean that and only that:

  • 50126, the submitted username or password was wrong. This is the guess.
  • 50056, invalid or missing password.
  • 50053, the account is locked because too many bad passwords were tried. This one is the guess having already done damage.

Gate the failure to those three and the Conditional Access and MFA policy noise is gone before any scoring happens. This single change removes the bulk of what made the rule loud, and it aligns the detection with the thing it claims to detect.

A Different IP Is Not Impossible Travel

The second problem is how it decides the two IPs are meaningfully different. It builds a block from the first two octets of the success IP and checks the failure does not start with it. That is a hand-rolled /16 that ignores CIDR entirely. It treats two of your own office ranges as different networks, and it treats an attacker who happens to share your first two octets as the same network. It is the same broken octet-prefix trick that shows up in a lot of community rules, and it should go.

Candidacy only needs the failure to come from a different address. The question of how suspicious that is should be scored, and the strongest signal for it is already in the data and never used: the location. The success carries a country and so does the failure. A failure from a different country while the account is in active use is the real divergence. The rule had both fields and never compared them.

Score It, Do Not Gate on UEBA Alone

The original's only real precision comes from one line: where UEBARiskScore > riskScoreCutoff. Without UEBA the rule is raw join noise, and with UEBA it is a single opaque number deciding everything. That is fragile both ways.

Better to weight the evidence. Each signal that points at guessing adds to a score, and the rule fires above a threshold. Nothing short-circuits, every signal that is true contributes, so a weak hit stays low and a corroborated one climbs:

  • Country mismatch between the success and the failure: weight 3.
  • The account was locked by the attempts (50053): weight 3.
  • Entra ID Protection scored the failed attempt medium or high: weight 3.
  • A burst of failed attempts from the IP, not a one-off: weight 2.
  • The targeted account is privileged: weight 2.
  • UEBA investigation priority on that user and IP is elevated: weight 2.
  • The account is a guest: weight 1.

With a threshold of five, a single local credential failure from a second device scores nothing and stays silent. A privileged user whose phone retries an old password a few times scores four and still stays silent. A burst of wrong passwords from a different country, or a lockout from a foreign IP, or anything Entra already flagged as risky, clears the bar. UEBA is now one weighted input among several, so the rule works without it and is sharper with it.

The RiskIndicators Field and the Score

The query carries both. The numeric Score drives the firing decision, the sort order, and the alert severity. The RiskIndicators string carries the same signals in readable form for the analyst:

CountryMismatch | AccountLockout | HighRiskFailure | FailureBurst | PrivilegedAccount | UEBAElevated | GuestAccount

The combination to chase is CountryMismatch with AccountLockout and PrivilegedAccount: an admin account in active use, locked out by wrong passwords coming from another country.

The Blind Spot You Should Know About

This is a concurrent-guessing detection. It needs a success and a credential failure close together to correlate, so a slow brute force against an account nobody is currently using will not surface here, that is a pure failed-logon rule on its own. And the scoring deliberately does not fire on a single same-country credential failure, because that is indistinguishable from a fat-fingered password on a second device. The serious same-country cases still surface through the lockout, burst, and risk weights. If you want the quieter ones too, lower the threshold.

MITRE Mappings for the Updated Rule

Tactic: Credential Access, with Initial Access for the external attempt.

T1110.001 Password Guessing. An actor trying credentials against a known, active account, which is exactly what the credential-failure gate isolates.

T1110.003 Password Spraying. The same pattern at volume, the burst weight and a foreign source covering the distributed case.

T1078.004 Valid Accounts, Cloud Accounts. The success side, a legitimate cloud session in progress that the guessing is aimed at.

Rule Settings

Run query every 60 minutes. Lookup data from the last 14 days.

Alert per result. Group by Account entity with a 6 hour lookback.

Entity mapping:

  • Name to Account (Name), UPNSuffix to Account (UPNSuffix)
  • FailedIP to IP (Address)

Custom details to surface in the incident: RiskIndicators, Score, FailedAttempts, FailedCountry, SuccessCountries, FailedResultTypes, Apps, MaxInvestigationPriority, AssignedRoles, UserType.

KQL

// =====================================================================
// Successful Logon From One IP, Credential Failure From Another - Entra ID
// =====================================================================
// Description : A user signs in successfully to an app, then a credential-type sign-in failure
//               (50126 / 50056 / 50053) hits the same app from a different IP within a short
//               window. Suspicion is weighted into a score (country mismatch, lockout, sign-in
//               risk, failure burst, privilege, UEBA) rather than gated on a single UEBA cutoff,
//               and all failures are filtered to credential-guessing codes so Conditional Access
//               and MFA policy failures never enter.
// Type        : Detection
//
// Tables      : SigninLogs, AADNonInteractiveUserSignInLogs, IdentityInfo, BehaviorAnalytics
// Connectors  : Microsoft Entra ID (SignInLogs, NonInteractiveUserSignInLogs),
//               Microsoft Sentinel UEBA (IdentityInfo, BehaviorAnalytics)
// License     : Microsoft Sentinel; Microsoft Entra ID P1 (sign-in logs),
//               P2 recommended (sign-in risk, UEBA)
//
// Tuning      : - LogonDiff - how close the success and failure must be
//               - DetectionWindow - the sign-in slice to correlate; align to run frequency
//               - ExcludedApps - legacy-auth-heavy apps that throw benign 50126 from cached creds
//               - BurstThreshold - failed attempts from one IP that count as a burst
//               - UebaThreshold - InvestigationPriority (0-10) that counts as elevated
//               - W_* weights and ScoreThreshold - shift these to fit your environment
//               - The UEBA and IdentityInfo joins are optional; remove them if UEBA is not enabled
//
// Known FPs   : - A second device blocked by Conditional Access or MFA - excluded by the credential-code gate
//               - A user's own device retrying a cached old password - scores low, single local failure suppressed
//               - Legacy-auth clients with stale credentials - confine via ExcludedApps
//
// Author      : Bartosz Wysocki | https://www.itprofessor.cloud
// Version     : 1.0 | 2026-06-17
// =====================================================================
let DetectionWindow = 1h;
let LogonDiff = 10m;
let IdentityLookback = 14d;       // full directory resync cycle; guarantees complete identity enrichment
let ExcludedApps = dynamic(["Office 365 Exchange Online", "Skype for Business Online"]);
let CredentialFailures = dynamic(["50126", "50056", "50053"]);  // wrong password, null password, locked by bad passwords
let BurstThreshold = 5;           // failed credential attempts from the IP within the window
let UebaThreshold = 4;            // BehaviorAnalytics InvestigationPriority (0-10) treated as elevated
let SensitiveRoles = dynamic([
"Global Administrator",
"Privileged Role Administrator",
"Privileged Authentication Administrator",
"Security Administrator",
"Exchange Administrator",
"SharePoint Administrator",
"User Administrator",
"Intune Administrator",
"Application Administrator",
"Hybrid Identity Administrator"
]);
// Scoring weights - higher weight for the signals that most separate guessing from a fumbled login
let W_CountryMismatch = 3;
let W_AccountLockout  = 3;
let W_HighRiskFailure = 3;
let W_FailureBurst    = 2;
let W_Privileged      = 2;
let W_Ueba            = 2;
let W_Guest           = 1;
let ScoreThreshold    = 5;
// Success on an app, then a credential failure on the same app from a different IP within LogonDiff
let CorrelatedPairs = (tableName:string) {
    table(tableName)
    | where TimeGenerated > ago(DetectionWindow)
    | where AppDisplayName !in~ (ExcludedApps)
    | where ResultType == "0"
    | project SuccessTime = TimeGenerated, UserPrincipalName = tolower(UserPrincipalName),
              SuccessIP = IPAddress, SuccessCountry = tostring(LocationDetails.countryOrRegion), AppDisplayName
    | join kind=inner (
        table(tableName)
        | where TimeGenerated > ago(DetectionWindow)
        | where AppDisplayName !in~ (ExcludedApps)
        | where ResultType in (CredentialFailures)
        | project FailedTime = TimeGenerated, UserPrincipalName = tolower(UserPrincipalName),
                  FailedIP = IPAddress, FailedCountry = tostring(LocationDetails.countryOrRegion),
                  FailedLocation = Location, AppDisplayName,
                  FailedResultType = ResultType, FailedRisk = tostring(RiskLevelDuringSignIn)
    ) on UserPrincipalName, AppDisplayName
    | where SuccessTime < FailedTime and FailedTime - SuccessTime <= LogonDiff and FailedIP != SuccessIP
};
let Pairs = union isfuzzy=true
    CorrelatedPairs("SigninLogs"),
    CorrelatedPairs("AADNonInteractiveUserSignInLogs");
// Identity context (privilege / guest), keyed on UPN
let IdentityContext = IdentityInfo
    | where TimeGenerated > ago(IdentityLookback)
    | summarize arg_max(TimeGenerated, AccountDisplayName, Department, JobTitle, AssignedRoles, UserType, IsAccountEnabled) by AccountUPN = tolower(AccountUPN);
// UEBA investigation priority for the user acting from the failed IP
let UebaContext = BehaviorAnalytics
    | where TimeGenerated > ago(DetectionWindow)
    | where ActivityType in ("FailedLogOn", "LogOn")
    | where isnotempty(SourceIPAddress)
    | summarize MaxInvestigationPriority = max(InvestigationPriority) by BAUser = tolower(UserPrincipalName), BAIP = SourceIPAddress;
Pairs
| extend HighRiskFailureRow = FailedRisk in ("high", "medium")
| summarize
    SuccessTime = max(SuccessTime),
    FailedTime = max(FailedTime),
    FailedAttempts = dcount(FailedTime),
    HighRiskFailureCount = countif(HighRiskFailureRow),
    SuccessIPs = make_set(SuccessIP, 10),
    SuccessCountries = make_set(SuccessCountry, 10),
    FailedCountry = max(FailedCountry),
    FailedLocation = max(FailedLocation),
    Apps = make_set(AppDisplayName, 10),
    FailedResultTypes = make_set(FailedResultType, 5)
  by UserPrincipalName, FailedIP
| join kind=leftouter IdentityContext on $left.UserPrincipalName == $right.AccountUPN
| join kind=leftouter UebaContext on $left.UserPrincipalName == $right.BAUser and $left.FailedIP == $right.BAIP
| extend CountryMismatch = isnotempty(FailedCountry) and not(set_has_element(SuccessCountries, FailedCountry))
| extend IsAccountLockout = set_has_element(FailedResultTypes, "50053")
| extend HighRiskFailure = HighRiskFailureCount > 0
| extend FailureBurst = FailedAttempts >= BurstThreshold
| extend IsPrivileged = tostring(AssignedRoles) has_any (SensitiveRoles)
| extend UEBAElevated = isnotnull(MaxInvestigationPriority) and MaxInvestigationPriority >= UebaThreshold
| extend IsGuest = tostring(UserType) =~ "Guest"
| extend Score = toint(CountryMismatch) * W_CountryMismatch
               + toint(IsAccountLockout) * W_AccountLockout
               + toint(HighRiskFailure) * W_HighRiskFailure
               + toint(FailureBurst)    * W_FailureBurst
               + toint(IsPrivileged)    * W_Privileged
               + toint(UEBAElevated)    * W_Ueba
               + toint(IsGuest)         * W_Guest
| where Score >= ScoreThreshold
| extend RiskIndicators = trim(@"\s\|\s*$", strcat(
    iff(CountryMismatch, "CountryMismatch | ", ""),
    iff(IsAccountLockout, "AccountLockout | ", ""),
    iff(HighRiskFailure, "HighRiskFailure | ", ""),
    iff(FailureBurst, "FailureBurst | ", ""),
    iff(IsPrivileged, "PrivilegedAccount | ", ""),
    iff(UEBAElevated, "UEBAElevated | ", ""),
    iff(IsGuest, "GuestAccount | ", "")
))
| extend Name = tostring(split(UserPrincipalName, "@", 0)[0]), UPNSuffix = tostring(split(UserPrincipalName, "@", 1)[0])
| project
    SuccessTime, FailedTime, Score, RiskIndicators,
    UserPrincipalName, Name, UPNSuffix,
    FailedIP, FailedCountry, FailedLocation, SuccessIPs, SuccessCountries,
    FailedAttempts, FailedResultTypes, Apps,
    MaxInvestigationPriority, AccountDisplayName, Department, JobTitle, AssignedRoles, UserType
| sort by Score desc

Follow my repo - GitHub

What You Should Do Next

  1. Run it over the last day or two in the Logs blade and read what scores highest. Tune the weights and ScoreThreshold down until the noise is gone. For testing only you can widen DetectionWindow past the rule's limit in Logs, the deployed rule still caps at the lookup period.
  2. Curate ExcludedApps. Legacy-auth-heavy apps throw benign 50126 from clients holding a cached old password, and that is where same-app credential-failure noise concentrates. Add the offenders you see.
  3. Decide whether same-country guesses matter to you. The score is built to suppress a single local failure. If you want those too, lower the threshold or raise the weights on the signals you trust.
  4. Wire severity to the score. Map AlertSeverity through dynamic alert details so a foreign lockout against an admin pages and a borderline hit does not, instead of one flat severity for everything.

Class dismissed