Watchlists in Microsoft Sentinel

Watchlists in Microsoft Sentinel allow you to correlate data from a data source you provide with the events in your Microsoft Sentinel environment. For example, you might create a watchlist with a list of high-value assets, terminated employees, or service accounts in your environment.

Use watchlists in your search, detection rules, threat hunting, and response playbooks.

Watchlists are stored in your Microsoft Sentinel workspace as name-value pairs and are cached for optimal query performance and low latency.

When to use watchlists

Use watchlists to help you with following scenarios:

• Investigate threats and respond to incidents quickly with the rapid import of IP addresses, file hashes, and other data from CSV files. After you import the data, use watchlist name-value pairs for joins and filters in alert rules, threat hunting, workbooks, notebooks, and general queries.

• Import business data as a watchlist. For example, import user lists with privileged system access, or terminated employees. Then, use the watchlist to create allowlists and blocklists to detect or prevent those users from logging in to the network.

• Reduce alert fatigue. Create allowlists to suppress alerts from a group of users, such as users from authorized IP addresses that perform tasks that would normally trigger the alert. Prevent benign events from becoming alerts.

• Enrich event data. Use watchlists to enrich your event data with name-value combinations derived from external data sources.

Limitations of watchlists

Before you create a watchlist, be aware of the following limitations:

• When you create a watchlist, the watchlist name and alias must each be between 3 and 64 characters. The first and last characters must be alphanumeric. But you can include whitespaces, hyphens, and underscores in between the first and last characters.
• The use of watchlists should be limited to reference data, as they aren't designed for large data volumes.
• The total number of active watchlist items across all watchlists in a single workspace is currently limited to 10 million. Deleted watchlist items don't count against this total. If you require the ability to reference large data volumes, consider ingesting them using custom logs instead.
• Watchlists are refreshed in your workspace every 12 days, updating the TimeGenerated field.
• Using Lighthouse to manage watchlists across different workspaces is not supported at this time.
• Local file uploads are currently limited to files of up to 3.8 MB in size.
• File uploads from an Azure Storage account (in preview) are currently limited to files up to 500 MB in size.
• Watchlists must adhere to the same column and table restrictions as KQL entities. For more information, see KQL entity names.

Options to create watchlists

Create a watchlist in Microsoft Sentinel from a file you upload from a local folder or from a file in your Azure Storage account.

You have the option to download one of the watchlist templates from Microsoft Sentinel to populate with your data. Then upload that file when you create the watchlist in Microsoft Sentinel.

To create a watchlist from a large file that's up to 500 MB in size, upload the file to your Azure Storage account. Then create a shared access signature URL for Microsoft Sentinel to retrieve the watchlist data. A shared access signature URL is an URI that contains both the resource URI and shared access signature token of a resource like a csv file in your storage account. Finally, add the watchlist to your workspace in Microsoft Sentinel.

For more information, see the following articles:

• Create watchlists in Microsoft Sentinel
• Built-in watchlist schemas
• Azure Storage SAS token

Watchlists in queries for searches and detection rules

Query data in any table against data from a watchlist by treating the watchlist as a table for joins and lookups. When you create a watchlist, you define the SearchKey. The search key is the name of a column in your watchlist that you expect to use as a join with other data or as a frequent object of searches. For example, suppose you have a server watchlist that contains country names and their respective two-letter country codes. You expect to use the country codes often for searches or joins. So you use the country code column as the search key.

The following example query joins the RemoteIPCountry column in the Heartbeat table with the search key defined for the watchlist named mywatchlist.

   Heartbeat
  | lookup kind=leftouter _GetWatchlist('mywatchlist') 
   on $left.RemoteIPCountry == $right.SearchKey

Let's look some other example queries.

Suppose you want to use a watchlist in an analytics rule. You create a watchlist called ipwatchlist that includes columns for IPAddress and Location. You define IPAddress as the SearchKey.

To only include events from IP addresses in the watchlist, you might use a query where watchlist is used as a variable or where the watchlist is used inline.

The following example query uses the watchlist as a variable:

  //Watchlist as a variable
  let watchlist = (_GetWatchlist('ipwatchlist') | project IPAddress);
  Heartbeat
  | where ComputerIP in (watchlist)

The following example query uses the watchlist inline with the query and the search key defined for the watchlist.

  //Watchlist inline with the query
  //Use SearchKey for the best performance
  Heartbeat
  | where ComputerIP in ( 
      (_GetWatchlist('ipwatchlist')
      | project SearchKey)
  )

For more information, see Build queries and detection rules with watchlists in Microsoft Sentinel.

Next steps

To learn more about Microsoft Sentinel, see the following articles:

• Create watchlists
• Build queries and detection rules with watchlists
• Manage watchlists
• Get started detecting threats with Microsoft Sentinel.
• Use workbooks to monitor your data.