User engagement data
Learn how to interpret app usage data and the factors that can impact data quality and consistency.
Trelica helps you to identify the apps that are in use in your organization and provides you with information so that you can make informed decisions about licensing, information security and data privacy. As part of this, Trelica collects data about which individuals are accessing apps and how frequently they log in to them.
You can use app user engagement data to determine whether to renew licenses or consolidate on a particular provider or product. Before doing so, it's important to understand the factors that can affect the consistency and quality of user engagement data.

Viewing user engagement data

User engagement data is only available for apps with a direct integration to Trelica, apps to which users log in via SAML2 Single Sign-On (SSO), and apps visited by users with the browser extension running.
When this login data is available, a summary of user engagement is displayed on the "Overview" tab for each app. By default, users that have not logged in to the app in the last 90 days are treated as unused accounts.
You can select a different threshold for unused accounts from the dropdown list (or change the default threshold for all apps as described below). Users that have logged in to the app within the threshold period are given a user engagement score of low, medium or high.
A summary of user engagement data is displayed on the "Overview" tab for each app.
When login data is available, you can view individuals' last login dates from the "Users" tab for each app. Where app login is via OpenID Connect, the "Last authenticated" date is displayed instead.
App users' last login dates are displayed on the "Users" tab for each app.
πŸ§™πŸΏβ€β™‚οΈYou can also compare user engagement across apps from the User engagement report.

Defining low, medium and high engagement

Initially, user engagement is calculated based on the recency of each user's last login, with what constitutes high, medium or low engagement varying according to the unused account threshold selected. For example, if the threshold for unused accounts is set to 30 days, a user whose last login was over 15 days ago (but within the last 30) will be categorized as low. By contrast, if the threshold is set to 90 days, a user is only categorized as low if they last logged in between 45 and 90 days ago.
As Trelica collects more login data for each app over time, we use this information to provide a more accurate assessment of user engagement. Once the login data covers the selected threshold period, the categorization changes to be based on the number of daily logins each user has made within that period. The lower the threshold, the more frequently a user needs to have logged in to be categorized as high.
The method that is used depends on the amount of data available. For example, if Trelica has login data for an app covering the last 45 days, setting the unused account threshold to 30 days will calculate user engagement based on the number of daily logins in the last 30 days. However, setting the threshold to 60 days will calculate user engagement based on recency of the last login only.

Changing the default user engagement threshold

By default, a user account is treated as "unused" if the individual's last login was over 90 days ago. To change this default for all apps:
  1. 1.
    Open Admin > Settings > Applications to open the Applications Settings page.
  2. 2.
    Click Unused account period to expand the section.
  3. 3.
    Use the dropdown list to select a different period. The user engagement summary is updated for each app for which the data is available.
The "Unused account period" setting.

Sources of user engagement data

As explained above, user engagement data is only available for apps with a direct integration to Trelica and for apps using "Big Bang" SAML2 SSO. The source of the data has an impact on consistency and overall quality.

Direct app integrations

Many apps record when a user last logged in and often this information is accessible through an API. If you connect Trelica directly to an app, we can extract this information to show how often users are logging in. A direct connection is the "gold standard" for usage data.

The impact of session duration

Some apps let users log in and remain logged in for a very long time. This is known as a long "session duration".
Long session durations are generally seen as a security weakness and are discouraged, but some apps trade off security best practice against user convenience.
Slack is a good example where, once installed, users are rarely asked to log in again. Where possible, Trelica will request a "last accessed" date, rather than a "last login" date, and display this information in the "Last login" column. This is what we do with Slack.
A different example would be Monday.com. By default, Monday.com has a long session duration, and the API only returns last login dates. This means that the last login date displayed in Trelica may not accurately reflect the last time the user accessed the app. Fortunately, the session duration in the Enterprise edition of Monday.com is configurable.
Where possible, we generally recommend reducing session durations to a maximum of 24 hours. This is both better practice from a security perspective, and provides more accurate usage data to Trelica.

"Big Bang" SAML2 SSO Apps

"Big Bang" refers to an app where users can only log in using SAML2 Single Sign-On (SSO); there is no way to log in to the app directly.
The benefit of apps where users are forced to log in via SAML2 is that Trelica can be sure that all logins to the app can be retrieved from the Identity Provider used for SSO (such as Okta, Google Workspace or Azure AD).
It is up to each app whether to implement a mechanism to disable password access, and force users to use SSO. For example Box.com has an "SSO required" checkbox.
How Big Bang SSO works

Non-"Big Bang" SAML2 SSO apps

If there is no direct app integration from Trelica, and if users can log in to the app with both passwords and with SSO, then while we can get some SSO data from the Identity Provider, we can't be sure to have captured all access.
The potential impact of this is that some users may be logging in, but we have no record of this in Trelica. In this case, the user engagement data only reflects the usage by users that have logged in to the app using SAML2 SSO.

OpenID Connect or "social" login access

"Sign in with Google" and "Sign in with Microsoft" buttons are both examples of the OpenID Connect protocol (the foundation for the OAuth2 protocol, also known as "social" login).
Although Google and Azure AD track "social" logins, this data does not provide an accurate representation of user engagement. The OpenID Connect protocol grants users an access token that can be valid for a long time (several months). This means that users might be accessing an app without actually having to log in again until the token has expired.
As OAuth2 / OpenID Connect data cannot be viewed as a reliable indication of access frequency, Trelica does not display user engagement data for apps discovered only via an OAuth2 integration. However, you can view the "Last authenticated" date for each user from the "Users" tab for each app; this indicates the last time the access token was granted.
Copy link
Outline
Viewing user engagement data
Defining low, medium and high engagement
Changing the default user engagement threshold
Sources of user engagement data
Direct app integrations
"Big Bang" SAML2 SSO Apps
Non-"Big Bang" SAML2 SSO apps
OpenID Connect or "social" login access