Read: 18 mins.
Print Friendly, PDF & Email

Locked Out?

If you are in Information technology, yous are more than than likely enlightened of issues that can ascend from business relationship lockouts, especially on a service account in use by a critical application or infrastructure component.

I volition dive into why lockouts occur, share troubleshooting steps, expect at helpful tools, and guide y'all into interpreting logs and so that the problem can be resolved every bit quickly as possible. A lockout tin prevent you, an awarding, or the business from standing piece of work. So, how exercise you go well-nigh in finding the source of the lockout?

Related: Visualize Business relationship Lockout events with my AD Lockout Splunk Dashboards to graphically identify patterns. For investigating Grouping -related events, meet my Group and Membership Changes post.

Bonus

As an added bonus, I have included information on how to expect up when an account was modified, disabled, enabled, unlocked, countersign reset — and past whom.

Active Directory Accounts

Allow's look at what Agile Directory is and how network logins are related.

Microsoft'south Agile Directory (Ad) is a service that governs how resource can exist utilized by a collection of users, groups, and computers. Enterprises utilise AD to authenticate, authorize, secure, and audit admission inside a security boundary — a Domain — to file servers, computers, emails, and more. You are given a user business relationship (often referred to equally your "network login") to access what has been made bachelor to you. A Domain Controller (DC) is the server that contains a re-create of the AD database and is responsible for the replication of said information between all other DCs within the Domain.

To secure the company network, Agile Directory uses Grouping Policy Objects (GPOs) to ascertain various user- and computer-related settings, including password policies and the Business relationship Lockout Threshold. The latter controls when an business relationship is locked after a set number of failed login attempts. If y'all mistype your password three times, for example, it would be locked for a specified fourth dimension or until an ambassador unlocks. This is an important security step to frustrate an unauthorized person from gaining admission.

What Happens During a Lockout?

Backside the scenes, when an wrong countersign was provided for an account, the Domain Controller that it authenticated to relays the request to the DC holding the "PDC Emulator" role. The PDC Emulator always holds the account's most recent password, then it volition re-cheque the provided password against its own database. If it is still incorrect, the PDC Emulator increments the badPwdCount attribute of the account, and an invalid login is recorded to a Security Event Log. If the badPwdCount has met the Account Lockout Threshold, the DC will lock the account, record Outcome ID 4740 (more than on that later) to its Security log, and notify the other Domain Controllers of the locked state. The key here is that every lockout is known by the PDC Emulator.

The PDC Emulator can be found through multiple ways, and my preferred one is via the control line:

How to notice the PDC Emulator of a Domain

  • Command Prompt: nltest /dclist:proper name (where name is the Ad Domain name)
  • PowerShell 
# Become the PDC Emulator for the current AD domain (Get-ADDomain).PDCEmulator # Get the PDC Emulator for "proper noun" (either Advertizing Domain or Domain Controller) (Get-ADDomain -Server name).PDCEmulator

Business relationship Lockout Threshold

Hither are two ways to apace notice the configured, Domain-wide threshold. A value of 0 means the account will never be locked. This setting can exist from 0 to 999.

  • Command Prompt: dsquery * -filter "(objectCategory=domain)" -attr lockoutThreshold
  • PowerShell
    • Get-ADDomain -Server domain | Select -ExpandProperty distinguishedName | Get-ADObject -Property lockoutThreshold
    • Go-ADDefaultDomainPasswordPolicy -Server domain

Warning: If your company uses Fine-Grained Password Policies (FGPP) introduced with Windows Server 2008, the to a higher place commands may not reverberate the password policy actually applied to individual accounts or Global Groups. FGPP allows an ambassador to set unlike password policies for various groups, such as privileged administrator accounts that crave more restrictive settings, than what is configured for the entire domain using GPO. FGPPs have precedence over the domain-wide GPO. Learn more

You may view a user's FGPP in PowerShell with: Go-ADUserResultantPasswordPolicy username. However, your ambassador may not accept granted rights to view information technology if the result ends with the fault, "Cannot observe an object with identity: 'CN=UserPasswordPolicy,CN=Password Settings Container,CN=System,DC=domain,DC=com".

Account Lockout Status

You lot can use Active Directory Users and Computers (ADUC) to cheque on an account's lockout status. However, for automation purposes, I prefer the control line:

To check lockout land:

  • Command Prompt: net user username /Domain
    • If "Account agile" is "No", it is locked or disabled
  •  PowerShell: Get-ADUser username -Properties LockedOut,AccountLockoutTime,badPwdCount

PowerShell: Lockout information for a item account

To unlock:

  • Command Prompt: net user username /Domain /Active:Yes
  • PowerShell
    • Get-ADUser username | Unlock-ADAccount
    • Unlock-ADAccount username

Troubleshooting

When I used to be in Desktop Support with big companies, I came across lockout tickets all the fourth dimension. Equally a Systems Engineer, I have more tools and access rights at my disposal to troubleshoot issues that were escalated to me. If after unlocking an account, the lockouts keep to occur, here are the general things I look at. Be sure to check both the local computer and whatever remote servers the account may have been used at.

General Areas

  1. Has the password changed recently?
    • Has the password been mistyped? I way to find out is to open a command prompt as that business relationship via RunAs. If there is no fault, the password is correct
  2. Is the account logged in elsewhere?
    • Look for disconnected or idle remote desktop/last server sessions
    • Bank check mobile devices, email clients in particular (Example: Kerberos Event ID 4771)
      • If your company uses Outlook Web Admission (OWA), look for whatever mobile devices associated with the email account
        • Become to OWA > Options > Mobile Devices
      • Reboot the device as needed
    • Have the user bank check their personal computers and devices

TIP: You can write a PowerShell script to gather all disconnected remote desktop/last server sessions from a list of computer names

  1. Is there a time of 24-hour interval blueprint?
    • Are lockouts happening merely during business hours? If so, it may bespeak to a device the user brings to piece of work, such as a phone, tablet, or laptop
  2. Saved WiFi password
    • If your company uses enterprise WiFi authentication, check the user device'due south WiFi password. Y'all may besides try turning off its WiFi for a few hours and see if that changes annihilation
  3. Bulldoze mappings
    • Disconnect all mapped drives and log out (or reboot). Whatsoever drives mapped by GPO would automatically reconnect upon login
  4. Scheduled tasks
    • Await for any tasks configured to run as the account
  5. Windows Credential Manager
    • Accept a look at the Web Passwords and the Windows Credentials sections
  6. Web browsers, such as Internet Explorer
    • Check saved credentials
    • Delete cookies and temporary files
      • As a last resort, endeavour deleting the browser history and all saved passwords
  7. Saved passwords in applications, including third political party software
    • With service accounts peculiarly, be sure the applications are using the correct password
    • If lockouts are occurring for multiple users from the aforementioned server, in that location may be a culprit application
    • Whatever applications that automatically log in
  8. Saved passwords on Windows Services
    • Go to services.msc and bank check any services that may be running nether the afflicted account
  9. Roaming contour may be corrupt
    • Accept the account logged off from every computer so the roaming contour is fully synchronized to the server
    • Rename the roaming profile on the server. We will be creating a NEW one with the side by side step
    • Every bit another user with admin rights, delete the roaming profile from One computer, then log on to it
      • A new roaming contour will be created on the figurer
        • Log off so it synchronizes a fresh copy to the server
      • If the result goes away, delete the roaming profiles on the other computers
      • Finally, manually copy any information from the erstwhile contour (on the server) dorsum to the new i
  10. Are an unusual number of other users likewise experiencing lockouts?
    • There may be a policy misconfiguration or the network is under set on
  11. Run an Antivirus scan
    • Has in that location been a worm infection recently? Worms are self-replicating malware that can use your credentials to travel beyond the network
  12. If all else fails, including going through the steps mentioned in the "Advanced Areas" section below, you may endeavour renaming the business relationship, but I generally suggest against that

Truthful story: In 2007, I was hunting a lockout for several days. I do not remember the details of how I institute the MAC address of the culprit, but I asked the network squad to check the switches to locate the port information technology came from. From the port, we traced to the connected computer, but oddly, there was no Windows profile for the affected account. As a event, that PC was initially dismissed, just lockouts kept coming from that address. Ultimately, I discovered that the account had a drive mapped under another user's profile. That person likely mapped a network share to get something, merely forgot to disconnect information technology when done.

Tip: If yous become a hold of a MAC address, you lot tin can try to look up the manufacturer here, here, or hither to help narrow down the hardware. The first 6 characters (example: Air conditioning:E2:D3) place the manufacturer. In the example, that would be "Hewlett Packard", my HP laptop.

Advanced Areas

  1. Agile Directory
    • Cheque the Account Lockout Threshold policy and encounter if it may be too restrictive, such every bit one that locks out subsequently just one failed endeavour
      • A Denial of Service (DoS) attack may take advantage of this to disrupt a business
    • Check replication functioning between affected sites. In that location may exist a delay in replicating countersign changes
    • Check the Security log with the Windows Consequence Viewer on Domain Controllers that have recorded Bad Password Counts, paying special attention to diverse Outcome IDs
      • A gratis tool from Microsoft can aid with gathering that data. More details on that and Event IDs later
  2. Enable Netlogon Debugging on the PDC Emulator for a few hours, if multiple users are affected, and await through the log
  3. Apply Netstat to await for any applications running on a server and disable any services or applications that may be trying to create a failed connection
  4. Inspect a doubtable computer'due south logons and processes
    • Should you have narrowed down the lockouts to come from a item computer, and none of the full general troubleshooting steps have helped, temporarily try logging some events locally
    • On the computer, with elevated administrator rights, run "gpedit.msc"
      • Go to Reckoner Configurations > Windows Settings > Security Settings > Local Policies > Audit Policy, and enable:
        1. Audit logon events: Success, Failure
        2. Audit process tracking: Success, Failure
      • Once the account locks once more, check the Event Viewer's Security log, paying attention to lockout-related Issue IDs that I will describe later in this post
        • Some of the events tin can requite y'all insight into what Caller Process or Source Network Address may have triggered the lockout
  5. Hacking attempts
    • Hackers may be trying to guess someone'due south countersign through diverse means, such equally a brute force attack
    • A disgruntled employee may exist locking his or her director's business relationship on purpose
  6. Penetration tests
    • An external company the security team has hired may exist performing a penetration test with unexpected results

True story: A penetration test happened at i of my employers during the middle of the day that inadvertently locked out multiple accounts, prompting our team to hunt downwards a possible hack endeavour. This brought to our attention that this type of Deprival of Service assail could easily disrupt a business if someone got a concord of the account names or login format

  1. Cheque the IIS Application Pool for any identities using the account
  2. Consider writing a script to temporarily purchase you more time to troubleshoot a business-critical service account, especially if your free energy was waning after troubleshooting all dark
    • Start with the "Account Lockout Status" section above to develop your script

True story: Recently, a service account used for bankroll up virtual machines, files, and more kept locking. I received an urgent Sun evening conference call at 7:30pm for assistance every bit backups were declining. After 4.5 hours of troubleshooting, checking Splunk logs, and using other methods, we were unable to find the lockout source. It was now midnight and some were tired.

  • Thankfully, I was writing a PowerShell script during the telephone call — just in case we needed to automate — that would cheque whether an business relationship was locked out and at what time it occurred. It would then give the pick to unlock in one case or to continuously monitor and automatically unlock. Ultimately, we used it so backups could complete while we got some residuum. (Exercise yous see why I prefer the control line and automating things?)
  • The next morning time, we found that the password was typed incorrectly on one of the fill-in servers

PowerShell: Account Lockout Check/Automatic Unlock

Don't forget: Wherever you had enabled additional debugging or auditing, remember to plow them off and then that performance does not suffer or space runs out!

Gathering Account Activity

As illustrated earlier, troubleshooting lockouts can exist a very fourth dimension consuming process. Non only are lockouts frustrating for the user, application owner, or even the business organisation, but the IT person tends to become the brunt of that fifty-fifty if it was not his or her fault. At that place are some tools yous could employ to help with troubleshooting, all of which essentially comb the Security section of the Windows Event Log. Here are some of my favorites.

Windows Event Viewer: Security Log

Free Tools

  1. Microsoft Account Lockout Status and EventCombMT
    • This is Microsoft's own utility
    • Lockoutstatus.exe: Displays the Bad Pwd Count, Final Bad Pwd date and time, when the password was last set, when the Lockout occurred, and which DC reported this data
    • EventCombMT
      • Can search through a listing of Domain Controllers for specific lockout-related Event IDs associated with the account. It will salve the output to a text file, showing successful and failed login attempts
        • Search for: 529, 644, 675, 676, 681, 4624, 4625, 4740 , 4770, 4771, 4776
        • Notation: see the "Interpreting Account Activeness" department below for Event ID details
      • Depending on your surround, this can take several hours to complete
  2. Netwrix Account Lockout Examiner
    • This is a popular, costless tool that gives you quite a lot of details. It besides performs a few, simple troubleshooting checks for y'all. However, I have tried the licensed version at two companies, and both times we ran into the same effect: information technology appears to have a memory leak that requires the server to exist rebooted from time to time

Scripts

  1. Issue-based Triggered Tasks
    • Windows provides a free, born way to perform certain tasks when a specific Event ID is recorded to the Outcome Log
      • If you remember, I mentioned before that the PDC Emulator records all lockout events. Use this fact to have the Domain Controller send you an electronic mail every time a lockout event (ID 4740) has occurred. This is achieved through an Outcome-based Triggered Task
    • When configured, you lot will see the trigger job listed in the DC's Task Scheduler
    • Having all the lockout emails stored in 1 folder allows you to search for a specific proper noun and quickly scan through the details
      • This saves you from having to search through multiple Domain Controllers' Event Logs

Tip: You can control what the email contains past writing a PowerShell script that formats and provides exactly the details that y'all may be interested in, including:
Username, Domain, Caller Car, Event ID, Lockout fourth dimension, Failure reason, Logon type, Caller Process Proper name, Source Network Accost, Source Port, and more

  1. PowerShell
    • You lot may realize past now that I am a supporter of PowerShell scripting. I have written a few of my own to assist with troubleshooting lockouts. Still, they are beyond the telescopic of this post, although I may share some in the futurity. Meanwhile, there are many, readily-available scripts at the PowerShell Gallery that y'all may desire to expect at.

Tool: Splunk ($)

Splunk is a VERY powerful, expensive tool that aggregates logs from multiple sources (such as systems, applications, network devices, and more) to allow you lot to search, monitor, and clarify a wealth of Large Data. It is a very useful SIEM (Security Data and Upshot Direction) tool that can as well be used to deconstruct a timeline of events, such as a breach in the network.

Splunk Search showing failed login attempts

I was first introduced to Splunk in tardily 2017 at one of my past employers, only did not accept the opportunity to endeavor it until this year. Knowing that the software aggregates Agile Directory logs at my electric current employer, I wanted to see how Splunk Cloud could be used to help with lockout troubleshooting. Its ambassador, however, was always too busy to help, and in that location were no good search queries I could discover on the spider web at the time. Naturally, I decided to learn some of the Splunk Search Processing Language (SPL) on my own to produce the queries presented in this article.

Tip: See the "Interpreting Account Activity" section below on how to read the data that Splunk and other tools produce

1 tin can turn SPL queries into beautiful Dashboards with Unproblematic XML like the ones I created below to help visualize lockout information:

Tip: See how we used Splunk Dashboards to visualize lockout data and chase for the causes in a MUCH shorter timeframe.

Consequence ID 4767 (Unlock) Notation

The Splunk queries provided here currently include events where the queried user is the i who performed an unlock operation. I have not yet added logic to exclude them. This tin can skew results when looking for events where the queried user itself was unlocked by someone else. In other words, the queries currently include events where either the user was unlocked or the user performed the unlock on another account.

Search for Lockout-related Events

This query will locate any events within the last 14 days that contribute to an account'southward lockout, including failed countersign attempts (and how the logon was used), which machine, IP address, and/or procedure spawned the result, and the actual lockout.

Lockout-related events

Lockout-related events

alphabetize="*" Account_Name=Michael.Yuen OR [email protected]* earliest=-14d latest=now() source=WinEventLog:Security (EventCode=4740 OR EventCode=4625 OR EventCode=644 OR EventCode=529 OR EventCode=675 OR EventCode=676 OR EventCode=681 OR EventCode=4771 OR EventCode=4770 OR EventCode=4768 OR EventCode=4776 OR EventCode=4777 OR EventCode=4725 OR EventCode=4723 OR EventCode=4724 OR EventCode=4767 OR EventCode=4800 OR EventCode=4801) | eval Account0=mvindex(Account_Name,0) | eval Account1=mvindex(Account_Name,1) | eval Account=case(EventCode==4624,Account1, EventCode==4625,Account1, EventCode==4648,Account1, EventCode==4722,Account1, EventCode==4723,Account1, EventCode==4724,Account1, EventCode==4725,Account1, EventCode==4738,Account1, EventCode==4740,Account1, EventCode==4767,Account1, EventCode==4768,Account0, EventCode==4769,Account0, EventCode==4771,Account0, EventCode==4770,Account0, EventCode==5140,Account0, EventCode==4778,Account0, EventCode==4779,Account0, EventCode==4800,Account0, EventCode==4801,Account0) | fillnull Value="-" Account | eval ActionBy=example(EventCode==4725,Account0, EventCode==4722,src_user, EventCode==4767,src_user, EventCode==4723,src_user, EventCode==4724,src_user, EventCode==4738,src_user, EventCode==4794,src_user) | eval Time=strftime(_time, "%m/%d/%y %H:%M:%South") | sort -_time | eval Caller_Machine=if(Caller_Machine_Name!= NULL, Caller_Machine_Name, Caller_Computer_Name) | fillnull Value="-" Caller_Machine | rex field=Process_Name "(?P<Process_Name>[^\\\]+)$" | fillnull Value="-" Process_Name | rex field=Caller_Process_Name "(?P<Caller_Process_Name>[^\\\]+)$" | fillnull Value="-" Caller_Process_Name | rename "Authentication_Package" as "Auth_Package", "Source_Network_Address" equally "Src_Netw_Addr" | supervene upon "MICROSOFT_AUTHENTICATION_PACKAGE_V1_0" with "MsAuthPkgV1_0" in Auth_Package | replace "Microsoft Unified Security Protocol Provider" with "MsUnifiedSecProt" in Auth_Package | eval EventCode=case(EventCode==4740, "4740 Locked", EventCode==4625, "4625 Logon Failed", EventCode==644, "644 Locked", EventCode==529, "529 Logon Failed", EventCode==4768, "4768 Kerb TGT Req", EventCode==4771, "4771 Kerb Pre-Auth Failed", EventCode==4770, "4770 Kerb Svc Tkt Renewed", EventCode==4624, "4624 Logon OK", EventCode==4648, "4648 Logon Attempt Explicit Creds (ie. Task/RunAs)", EventCode==4767, "4767 Unlocked", EventCode==12294, "12294 Potential attack against Administrator", EventCode==4794, "4794 DSRM Admin PW Gear up Endeavor", EventCode==4725, "4725 Disabled", EventCode==4722, "4722 Enabled", EventCode==4723, "4723 PW change attempt",EventCode==4724, "4724 PW reset attempt", EventCode==4738, "4738 Object changed", EventCode==4720, "4720 Created", EventCode==4726, "4726 Deleted", EventCode==4778, "4778 Session Reconnect", EventCode==4779, "4779 Session Disconnect", EventCode==4800, "4800 Wks Locked", EventCode==4801, "4801 Wks Unlocked", 1=one, EventCode) | eval Logon_Type=case(Logon_Type==two, "2 Interactive", Logon_Type==iii, "3 Network", Logon_Type==4, "iv Batch", Logon_Type==5, "5 Service", Logon_Type==vii, "7 Unlock", Logon_Type==8, "viii NetworkClearText", Logon_Type==9, "9 NewCredentials", Logon_Type==10, "10 RemoteInteractive", Logon_Type==11, "xi CachedInteractive", 1=1, Logon_Type) `comment(" | dedup Time, Business relationship | dedup Account, Caller_Machine ")` | table Fourth dimension, Account, EventCode, ActionBy, Caller_Machine, Failure_Reason, Client_Address, Logon_Type, Logon_Process, Auth_Package, Caller_Process_Name, Process_Name, ComputerName, Workstation_Name, Src_Netw_Addr `comment(" | stats count by Business relationship, Caller_Machine, EventCode | sort -count ")`

Tips and Notes:

  • Replace "index=*" with the appropriate index to search through. This tin significantly shorten search time
    • You lot tin find the listing of available indexes by querying: "| eventcount summarize=faux alphabetize=* index=_* | dedup alphabetize | fields index"
  • The portion "OR Account_Name=Mic[electronic mail protected]*" exists solely to gather information for Issue ID 4770 (Kerberos). It can be removed if Event 4770 is not needed
  • To summarize events equally striking counts, uncomment the "stats count" line
    • Can more chop-chop help in identifying which Caller_Machine/Events are most affected
  • Commands are case-sensitive
  • Replace "Account_name=" with "ComputerName=XYZ" [where "XYZ" is either a computer's FQDN or "Proper noun*" (wildcard)] to search for events logged by that machine
  • "ane=1, EventCode" means: Default = EventCode itself (to take hold of whatsoever value not matching any Case evaluations)
  • "mvindex(Account_Name,0)" means: Get the first value of "Account_Name" (to select from multiple values)

Search for Not-Lockout-related Events

This query will help effigy out where the account may accept been used successfully within the last 14 days, including logons, logoffs, and which machines, IP addresses, and/or processes contributed to the event. Optionally, it will as well evidence who unlocked the account.

Non-lockout related events

Non-lockout related events

alphabetize="*" Account_Name=Michael.Yuen OR [email protected]* earliest=-14d latest=now() source=WinEventLog:Security NOT (EventCode=4740 OR EventCode=4625 OR EventCode=644 OR EventCode=529 OR EventCode=675 OR EventCode=676 OR EventCode=681 OR EventCode=4771 OR EventCode=4770 OR EventCode=4768 OR EventCode=4776 OR EventCode=4777) NOT EventCode=4767 NOT (EventCode=6272 OR EventCode=6273 OR EventCode=6274 OR EventCode=6275) Non (EventCode=4627 OR EventCode=4780) | eval Account0=mvindex(Account_Name,0) | eval Account1=mvindex(Account_Name,1) | eval Account=case(EventCode==4624,Account1, EventCode==4625,Account1, EventCode==4648,Account1, EventCode==4722,Account1, EventCode==4723,Account1, EventCode==4724,Account1, EventCode==4725,Account1, EventCode==4738,Account1, EventCode==4740,Account1, EventCode==4767,Account1, EventCode==4768,Account0, EventCode==4769,Account0, EventCode==4771,Account0, EventCode==4770,Account0, EventCode==5140,Account0, EventCode==4778,Account0, EventCode==4779,Account0, EventCode==4800,Account0, EventCode==4801,Account0) | fillnull Value="-" Account | eval ActionBy=case(EventCode==4725,Account0, EventCode==4722,src_user, EventCode==4767,src_user, EventCode==4723,src_user, EventCode==4724,src_user, EventCode==4738,src_user, EventCode==4794,src_user) | eval Time=strftime(_time, "%1000/%d/%y %H:%K:%South") | sort -_time | eval Caller_Machine=if(Caller_Machine_Name!= NULL, Caller_Machine_Name, Caller_Computer_Name) | fillnull Value="-" Caller_Machine | king field=Process_Name "(?P<Process_Name>[^\\\]+)$" | fillnull Value="-" Process_Name | rex field=Caller_Process_Name "(?P<Caller_Process_Name>[^\\\]+)$" | fillnull Value="-" Caller_Process_Name | rename "Authentication_Package" every bit "Auth_Package", "Source_Network_Address" equally "Src_Netw_Addr" | replace "MICROSOFT_AUTHENTICATION_PACKAGE_V1_0" with "MsAuthPkgV1_0" in Auth_Package | replace "Microsoft Unified Security Protocol Provider" with "MsUnifiedSecProt" in Auth_Package | eval EventCode=case(EventCode==4740, "4740 Locked", EventCode==4625, "4625 Logon Failed", EventCode==644, "644 Locked", EventCode==529, "529 Logon Failed", EventCode==4768, "4768 Kerb TGT Req", EventCode==4771, "4771 Kerb Pre-Auth Failed", EventCode==4770, "4770 Kerb Svc Tkt Renewed", EventCode==4624, "4624 Logon OK", EventCode==4648, "4648 Logon Try Explicit Creds (ie. Task/RunAs)", EventCode==4767, "4767 Unlocked", EventCode==12294, "12294 Potential attack against Administrator", EventCode==4794, "4794 DSRM Admin Pow Set Attempt", EventCode==4725, "4725 Disabled", EventCode==4722, "4722 Enabled", EventCode==4723, "4723 Pw modify attempt",EventCode==4724, "4724 PW reset attempt", EventCode==4738, "4738 Object changed", EventCode==4720, "4720 Created", EventCode==4726, "4726 Deleted", EventCode==4778, "4778 Session Reconnect", EventCode==4779, "4779 Session Disconnect", EventCode==4800, "4800 Wks Locked", EventCode==4801, "4801 Wks Unlocked", one=1, EventCode) | eval Logon_Type=case(Logon_Type==2, "2 Interactive", Logon_Type==iii, "3 Network", Logon_Type==four, "4 Batch", Logon_Type==five, "five Service", Logon_Type==7, "7 Unlock", Logon_Type==8, "8 NetworkClearText", Logon_Type==ix, "9 NewCredentials", Logon_Type==10, "10 RemoteInteractive", Logon_Type==eleven, "11 CachedInteractive", ane=1, Logon_Type) | tabular array Time, Account, TaskCategory, EventCode, ActionBy, Logon_Type, Logon_Process, Auth_Package, Caller_Process_Name, Process_Name, ComputerName, Src_Netw_Addr, Network_Address

Notation: Remove "NOT EventCode=4767" in the query above, if you want to also see who unlocked the account.

Search for ALL Events related to the account

This query combines lockout-related and non-lockout-related events from the last 14 days for a ameliorate expect at the timeline leading up to a lockout.

All account-related events

All account-related events

index="*" Account_Name=Michael.Yuen OR [email protected]* primeval=-14d latest=now() source=WinEventLog:Security | eval Account0=mvindex(Account_Name,0) | eval Account1=mvindex(Account_Name,one) | eval Business relationship=instance(EventCode==4624,Account1, EventCode==4625,Account1, EventCode==4648,Account1, EventCode==4722,Account1, EventCode==4723,Account1, EventCode==4724,Account1, EventCode==4725,Account1, EventCode==4738,Account1, EventCode==4740,Account1, EventCode==4767,Account1, EventCode==4768,Account0, EventCode==4769,Account0, EventCode==4771,Account0, EventCode==4770,Account0, EventCode==5140,Account0, EventCode==4778,Account0, EventCode==4779,Account0, EventCode==4800,Account0, EventCode==4801,Account0) | fillnull Value="-" Account | eval ActionBy=case(EventCode==4725,Account0, EventCode==4722,src_user, EventCode==4767,src_user, EventCode==4723,src_user, EventCode==4724,src_user, EventCode==4738,src_user, EventCode==4794,src_user) | eval Fourth dimension=strftime(_time, "%thousand/%d/%y %H:%M:%Due south") | sort -_time | eval Caller_Machine=if(Caller_Machine_Name!= Nix, Caller_Machine_Name, Caller_Computer_Name) | fillnull Value="-" Caller_Machine | rex field=Process_Name "(?P<Process_Name>[^\\\]+)$" | fillnull Value="-" Process_Name | rex field=Caller_Process_Name "(?P<Caller_Process_Name>[^\\\]+)$" | fillnull Value="-" Caller_Process_Name | rename "Authentication_Package" equally "Auth_Package", "Source_Network_Address" as "Src_Netw_Addr" | supplant "MICROSOFT_AUTHENTICATION_PACKAGE_V1_0" with "MsAuthPkgV1_0" in Auth_Package | replace "Microsoft Unified Security Protocol Provider" with "MsUnifiedSecProt" in Auth_Package | eval EventCode=example(EventCode==4740, "4740 Locked", EventCode==4625, "4625 Logon Failed", EventCode==644, "644 Locked", EventCode==529, "529 Logon Failed", EventCode==4768, "4768 Kerb TGT Req", EventCode==4771, "4771 Kerb Pre-Auth Failed", EventCode==4770, "4770 Kerb Svc Tkt Renewed", EventCode==4624, "4624 Logon OK", EventCode==4648, "4648 Logon Effort Explicit Creds (ie. Job/RunAs)", EventCode==4767, "4767 Unlocked", EventCode==12294, "12294 Potential assault against Administrator", EventCode==4794, "4794 DSRM Admin Pw Prepare Attempt", EventCode==4725, "4725 Disabled", EventCode==4722, "4722 Enabled", EventCode==4723, "4723 Prisoner of war change endeavor",EventCode==4724, "4724 Pow reset endeavour", EventCode==4738, "4738 Object inverse", EventCode==4720, "4720 Created", EventCode==4726, "4726 Deleted", EventCode==4778, "4778 Session Reconnect", EventCode==4779, "4779 Session Disconnect", EventCode==4800, "4800 Wks Locked", EventCode==4801, "4801 Wks Unlocked", i=1, EventCode) | eval Logon_Type=case(Logon_Type==2, "2 Interactive", Logon_Type==3, "3 Network", Logon_Type==4, "4 Batch", Logon_Type==5, "v Service", Logon_Type==7, "seven Unlock", Logon_Type==eight, "8 NetworkClearText", Logon_Type==9, "9 NewCredentials", Logon_Type==10, "10 RemoteInteractive", Logon_Type==eleven, "11 CachedInteractive", i=1, Logon_Type) | table Time, Account, TaskCategory, EventCode, ActionBy, Caller_Machine, Failure_Reason, Client_Address, Logon_Type, Logon_Process, Auth_Package, Caller_Process_Name, Process_Name, ComputerName, Workstation_Name, Src_Netw_Addr, Network_Address

Bonus: Search for Account Modifications — and by Whom

This query searches through the terminal 30 days for specific business relationship modification events — and past whom, including: Disable, Enable, Unlock, Modify, Create, Delete, Password Reset

Account modifications and by whom

Account modifications and by whom

index="*" Account_Name=Michael.Yuen earliest=-30d latest=at present() source=WinEventLog:Security (EventCode=4725 OR EventCode=4722 OR EventCode=4724 OR EventCode=4738 OR EventCode=4767 OR EventCode=4720 OR EventCode=4726) | eval ActionBy=src_user | eval Business relationship=mvindex(Security_ID,1) | eval Time=strftime(_time, "%one thousand/%d/%y %H:%Grand:%S") | sort -_time | eval EventCode=instance(EventCode==4767, "4767 Unlocked", EventCode==4725, "4725 Disabled", EventCode==4722, "4722 Enabled", EventCode==4723, "4723 Pow change attempt", EventCode==4724, "4724 PW reset attempt", EventCode==4738, "4738 Object changed", EventCode==4720, "4720 Created", EventCode==4726, "4726 Deleted", 1=1, EventCode) | table Time, Account, EventCode, ActionBy, ComputerName

Now that Splunk has produced some reports, how exercise y'all interpret them? This will be covered in the adjacent department.

Tip: The Splunk results can be downloaded as CSV, XML, and JSON files for further reporting and assay. They can also be printed as PDF, although at the time of writing, the orientation is locked in Portrait mode

Interpreting Account Activity

The amount of data that tin exist collected well-nigh an business relationship's activity can be very overwhelming. "Event ID 4740" — what is that? What does "Logon Type 3" mean? What about "Network Address" and "Caller Machine"? What if a lockout shows the source to be a Domain Controller? Does that mean an unauthorized user attempted to log in to the DC, and if so, why?

Windows Event IDs

Before we can interpret the Splunk results (or output provided by Microsoft's EventCombMT and other tools), let's await at the various Event IDs that are related to account activeness.

In the above table, I have separated the events by two types: Lockout and Info. The former are general sources that could contribute to a lockout, and the latter are informational to assistance gain an understanding of what had occurred. The ones to pay particular attending to:

  • 4740: Business relationship has been locked
  • 4625: At that place was a failed logon attempt
Lockout-related events

Lockout-related events

When I troubleshoot an account, I look for a grouping of 4625 events that may take led to the 4740 lock event. The message within 4740 would tell you what ultimately caused the lock. Go on in heed that although 4625 reports a failed logon, it may not necessarily have been the culprit. It may have simply failed because the account was ALREADY locked as illustrated in the beneath 4625 sample.

Event ID 4740 - Locked

Event ID 4740 – Locked

Result ID 4740: The account, "DOMAIN\MichaelYuen" was locked out by "Caller Figurer Proper name", "MyComputer1". "DC01" logged this event. The lock likely came from "MyComputer1". Outset troubleshooting in that location.

If "Caller Estimator Proper noun" is blank/naught or a Domain Controller, look at the "Process Information" and "Network Information" fields side by side:

  • Process Information: the process on the computer that requested the logon
  • Network Information: the remote estimator that requested the logon

The side by side example provides more details on both.

Event ID 4625 - Logon Failure

Event ID 4625 – Logon Failure

Event ID 4625: "DC02" (from the Subject field) reported the logon failure for "Account Name: MichaelYuen" and cites "Failure Reason: Account locked out". The business relationship could not log in considering it was ALREADY locked.

  • The Caller Process, "lsass.exe" (Windows login service), on the remote computer with IP of 10.one.1.100 attempted the login. Perform a DNS/WINS lookup (example: "ping -a 10.1.1.100") to find the name of that computer/server and start troubleshooting there.
    • "svchost.exe" is used to launch Windows services
  • Under Network Information, theWorkstation Name is "DC02", a Domain Controller. It often as well tin be blank
    • In my experience, when the Caller Computer Name or Workstation Name are either bare or a DC, the request probable came from a non-Windows machine, such as a Linux/Unix server or an appliance based on those operating systems (ie. F5 Big-IP and Citrix Netscaler load balancers or a VMware Host)
  • Logon Type is listed as "3". That is another clue we tin employ to notice out how the offending computer or procedure tried to log in
  • Exist sure to also take a wait at the Logon Process and Hallmark Packet that will exist discussed later

Logon Types

Logon Type is the method an account tried to log in with. Let's look at the various types.

In the example I had provided for Event ID 4625, the Logon Type was "3". The logon was attempted over the network from a remote server — a VMware Host in that particular situation.

Lockout-related events

Lockout-related events

Logon Process, Authentication Parcel

When I created a Splunk Dashboard to visualize and analyze data, I noticed a large amount of Event ID 4624 (Successful logon) for a user. That led me to look into its "Detailed Authentication Information" section to see where the logins were coming from. Of note were Logon Process and Authentication Package that apply to Issue IDs 4624 and 4625 (Failed logon).

The login process starts with credential collection that is passed on to the Local Security Authorization (LSA) Server Service (LSASS) or Service Host (SvcHost). It after works with a Logon Process (a Windows DLL, Dynamic Link Library) to handle the actual authentication try with an Authentication Package.

Event ID 4624/4625 - Logon Process, Authentication Package

Consequence ID 4624/4625 – Logon Procedure, Authentication Package

Every bit I understand it… LSASS or SvcHost initiates the logon, and Logon Procedure processes the authentication with an Authentication Package type.

Logon Processes

The Logon Process works with an Authentication Package to handle the actual authentication started past the LSASS or SvcHost process.

Authentication Packages

The Authentication Package is used by the Logon Procedure to handle the bodily authentication try.

Kerberos is the default authentication package that replaced NTLM (NT LAN Manager) since Windows Server 2000. If Kerberos fails to authenticate the user, NTLM will be used instead.

Kerberos Authentication Process /ManageEngine

Kerberos Authentication Process /ManageEngine

Final Thoughts

Looking for the suspect procedure, application, or computer can exist a examination of patience and perseverance, and having customers understandably vent their frustration in an effort to nudge you to find the source can be discouraging. Hopefully, the data and tools presented here volition aid you towards a quick(er) resolution, and helped you lot understand how Active Directory tracks account activities. Furthermore, I encourage you to learn PowerShell, if yous haven't already, equally information technology tin be a very powerful tool to have in your Information technology arsenal.

Happy lockout hunting!

Related: Visualize Account Lockout events with my AD Lockout Splunk Dashboards to graphically identify patterns. For investigating Grouping -related events, run into my Group and Membership Changes mail.

Additional References

  • EventSentry: Event IDs
  • Microsoft Docs: Windows Logon Scenarios / Inspect Logon
  • Randy Franklin Smith's "Ultimate It Security": Logon/Logoff Events / Account Direction Events
  • Pearson's Microsoft Printing Store: Windows Logon
  • Crowdstrike: NTLM vs Kerberos
  • ManageEngine: Kerberos Hallmark Ticket Request (Event ID 4768)

Related Posts

  • Active Directory
    • Visualize Account Lockouts with Splunk Dashboards
    • Group and Membership Changes
  • Azure Active Directory
    • Audit Office Changes (ie. Global Administrator)
    • Successful Logins from Unknown Users

Credits:
– Featured Image by Warren Wong via Unsplash