Troubleshooting: Realms Load or Authenticate Slowly

Version Affected: All
 
 

Overview

After confirming there are no problems with the realm's data source (Active Directory, OpenLDAP, SQL, etc.) and no other server-side issue with SecureAuth, realms can still load or authenticate very slowly. If neither cause below matches, general performance tuning may help instead -- see How To: Speed Up Active Directory Lookups and How To: Tune IIS Application Pool Performance and Diagnose Slow or Failed Authentication Requests.

• See Cause 1 - AWS QuickSight or FIDO Service Endpoint Unreachable (24.04 and Later)
• See Cause 2 - Logging Server or trx.secureauth.com Connectivity Issue

These causes are not related, so a fix for one cause will not resolve the others.

 

In this article

• Cause 1: AWS QuickSight or FIDO Service Endpoint Unreachable (24.04 and Later)
• Cause 2: Logging Server or trx.secureauth.com Connectivity Issue

Cause 1: AWS QuickSight or FIDO Service Endpoint Unreachable (24.04 and Later)

The realm's Audit log shows timeouts for both the mobile service and the FIDO service, similar to the following (trimmed to the relevant fields):

LogChannel="SA_AUDIT" EventID="94010" Realm="SecureAuth51" Message="Mobile Service GetDevices, Status: Timeout reached. ResponseTime='11106'"
LogChannel="SA_AUDIT" EventID="70050" Realm="SecureAuth51" Message="Fido Service GetTokens, Status: Timeout reached. ResponseTime='10006'"

Checking the appliance's most recent nlog for the same time period shows it's unable to reach us-east-1.quicksight.aws.amazon.com -- a new AWS endpoint required starting with IdP 24.04. Without being able to reach it (and the other endpoint documented in the resolution below), each affected request waits out its full timeout before continuing, and those timeouts add up to the high latency observed during authentication.

Resolution 1:

To resolve this:

Ensure all required network endpoints are accessible, specifically us-east-1.quicksight.aws.amazon.com, which is required for this issue to resolve. SecureAuth's Cloud Services documentation lists the current complete set of endpoints required for IdP 24.04 and later -- whitelist all of them, not just the one above, since any of them being unreachable can produce the same symptom. Once every required endpoint is reachable, authentication times return to normal.

Cause 2: Logging Server or trx.secureauth.com Connectivity Issue

SecureAuth logs and authenticates on a single CPU thread, so anything that slows down or blocks that thread slows down every realm's authentication along with it. Two common culprits are the logging server itself, and connectivity to trx.secureauth.com: if a SQL server or SIEM used for log storage responds slowly or isn't reachable, every authentication waits on it; separately, the appliance periodically checks connectivity to trx.secureauth.com, and if that check can't complete, the system behaves poorly while it waits.

Resolution 2:

To resolve this:

1. If using a SQL server for log data, go to the Logs tab and click the test button to confirm connectivity. If the test fails, check firewall permissions between the IdP and the SQL server.
2. Check whether the SQL logging database is full, and check its performance counters for over-utilization -- especially if the same SQL server is shared with other applications.
3. If the SQL logging server or SIEM is at a different site than the IdP, consider moving it to the same site to reduce latency.
4. For the trx.secureauth.com connectivity issue, resolve the underlying network problem. Ping won't work for this check -- open a browser and confirm http://trx.secureauth.com responds.

SecureAuth Knowledge Base Articles provide information based on specific use cases and may not apply to all appliances or configurations. Be advised that these instructions could cause harm to the environment if not followed correctly or if they do not apply to the current use case.

Customers are responsible for their own due diligence prior to utilizing this information and agree that SecureAuth is not liable for any issues caused by misconfiguration directly or indirectly related to SecureAuth products.