Insights and reports for Attack simulation training

Tip

Did you know you can try the features in Microsoft Defender XDR for Office 365 Plan 2 for free? Use the 90-day Defender for Office 365 trial at the Microsoft Defender portal trials hub. Learn about who can sign up and trial terms here.

In Attack simulation training in Microsoft Defender for Office 365 Plan 2 or Microsoft 365 E5, Microsoft provides insights and reports from the results of simulations and the corresponding trainings. This information keeps you informed on the threat readiness progress of your users, and recommended next steps to better prepare your users for future attacks.

Insights and reports are available in the following locations on the Attack simulation training page in the Microsoft Defender portal:

The rest of this article describes the reports and insights for Attack simulation training.

For getting started information about Attack simulation training, see Get started using Attack simulation training.

Insights on the Overview and Reports tabs of Attack simulation training

To go to the Overview tab, open the Microsoft Defender portal at https://security.microsoft.com, go to Email & collaboration > Attack simulation training:

The distribution of insights on the tabs is described in the following table:

Report Overview tab Reports tab
Recent simulations card
Recommendations card
Simulation coverage card
Training completion card
Repeat offenders card
Behavior impact on compromise rate card

The rest of this section describes the information that's available on the Overview and Reports tabs of Attack simulation training.

Recent simulations card

The Recent simulations card on the Overview tab shows the last three simulations that you created or ran in your organization.

You can select a simulation to view details.

Selecting View all simulations takes you to the Simulations tab.

Selecting Launch a simulation starts the new simulation wizard. For more information, see Simulate a phishing attack in Defender for Office 365.

The Recent simulations card on the Overview tab in Attack simulation training in the Microsoft Defender portal.

Recommendations card

The Recommendations card on the Overview tab suggests different types of simulations to run.

Selecting Launch now starts the new simulation wizard with the specified simulation type automatically selected on the Select technique page. For more information, see Simulate a phishing attack in Defender for Office 365.

The Recommendations card on the Overview tab in Attack simulation training in the Microsoft Defender portal.

Simulation coverage card

The Simulation coverage card on the Overview and Reports tabs shows the percentage of users in your organization who received a simulation (Simulated users) vs. users who didn't receive a simulation (Non-simulated users). You can hover over a section in the chart to see the actual number of users in each category.

Selecting View simulation coverage report takes you to the User coverage tab for the Attack simulation report.

Selecting Launch simulation for non-simulated users starts the new simulation wizard where the users who didn't receive the simulation are automatically selected on the Target user page. For more information, see Simulate a phishing attack in Defender for Office 365.

The Simulation coverage card on the Overview tab in Attack simulation training in the Microsoft Defender portal.

Training completion card

The Training completion card on the Overview and Reports tabs organizes the percentages of users who received trainings based on the results of simulations into the following categories:

  • Completed
  • In progress
  • Incomplete

You can hover over a section in the chart to see the actual number of users in each category.

Selecting View training completion report takes you to the Training completion tab for the Attack simulation report.

The Training completion card on the Overview tab in Attack simulation training in the Microsoft Defender portal.

Repeat offenders card

The Repeat offenders card on the Overview and Reports tabs shows the information about repeat offenders. A repeat offender is a user who was compromised by consecutive simulations. The default number of consecutive simulations is two, but you can change the value on the Settings tab of Attack simulation training at https://security.microsoft.com/attacksimulator?viewid=setting. For more information, see Configure the repeat offender threshold.

The chart organizes repeat offender data by simulation type:

  • All
  • Malware Attachment
  • Link to Malware
  • Credential Harvest
  • Link in attachments
  • Drive-by URL

Selecting View repeat offender report takes you to the Repeat offenders tab for the Attack simulation report.

The Repeat offenders card on the Overview tab in Attack simulation training in the Microsoft Defender portal

Behavior impact on compromise rate card

The Behavior impact on compromise rate card on the Overview and Reports tabs shows how your users responded to your simulations as compared to the historical data in Microsoft 365. You can use these insights to track progress in users threat readiness by running multiple simulations against the same groups of users.

The chart data shows the following information:

  • Actual compromise rate: The actual percentage of people who were compromised by the simulation (actual users compromised / total number of users in your organization who received the simulation).
  • Predicted compromise rate: Historical data across Microsoft 365 that predicts the percentage of people who will be compromised by this simulation. To learn more about the predicted compromise rate (PCR), see Predicted compromise rate.

If you hover over a data point in the chart, the actual percentage values are shown.

To see a detailed report, select View simulations and training efficacy report. This report is explained later in this article.

The Behavior impact on compromise rate card on the Overview tab in Attack simulation training in the Microsoft Defender portal.

Attack simulation report

You can open the Attack simulation report from the Overview tab by selecting the View ... report actions that are available on some of the cards on the Overview and Reports tabs that are described in this article. To go directly to the Attack simulation report page, use https://security.microsoft.com/attacksimulationreport

Training efficacy tab for the Attack simulation report

The Training efficacy tab is selected by default on the Attack simulation report page. This tab provides the same information that's available in the Behavior impact on compromise rate card, with additional context from the simulation itself.

The Training efficacy tab in the Attack simulation report in the Microsoft Defender portal.

The chart shows the Actual compromised rate and the Predicted compromise rate. If you hover over a section in the chart, the actual percentage values for are shown.

The details table below the chart shows the following information. You can sort the simulations by clicking on an available column header. Select Customize columns to change the columns that are shown. By default, all available columns are selected.

  • Simulation name
  • Simulation technique
  • Simulation tactics
  • Predicted compromised rate
  • Actual compromised rate
  • Total users targeted
  • Count of clicked users

Use the Search box to filter the results by Simulation name or Simulation Technique. Wildcards aren't supported.

Use the Export report button to save the information to a CSV file. The default filename is Attack simulation report - Microsoft Defender.csv, and the default location is the local Downloads folder. If an exported report already exists in that location, the filename is incremented (for example, Attack simulation report - Microsoft Defender (1).csv).

User coverage tab for the Attack simulation report

On the User coverage tab, the chart shows the Simulated users and Non-simulated users. If you hover over a data point in the chart, the actual values are shown.

The User coverage tab in the Attack simulation report in the Microsoft Defender portal.

The details table below the chart shows the following information. You can sort the information by clicking on an available column header. Select Customize columns to change the columns that are shown. By default, all available columns are selected.

  • Username
  • Email address
  • Included in simulation
  • Date of last simulation
  • Last simulation result
  • Count of clicked
  • Count of compromised

Use the Search box to filter the results by Username or Email address. Wildcards aren't supported.

Use the Export report button to save the information to a CSV file. The default filename is Attack simulation report - Microsoft Defender.csv, and the default location is the local Downloads folder. If an exported report already exists in that location, the filename is incremented (for example, Attack simulation report - Microsoft Defender (1).csv).

Training completion tab for the Attack simulation report

On the Training completion tab, the chart shows the number of Completed, In progress, and Incomplete simulations. If you hover over a section in the chart, the actual values are shown.

The Training completion tab in the Attack simulation report in the Microsoft Defender portal.

The details table below the chart shows the following information. You can sort the information by clicking on an available column header. Select Customize columns to change the columns that are shown. By default, all available columns are selected.

  • Username
  • Email address
  • Included in simulation
  • Date of last simulation
  • Last simulation result
  • Name of most recent training completed
  • Date completed
  • All trainings

Select Filter to filter the chart and details table by a Status values of the trainings: Completed, In progress, or All.

When you're finished configuring the filters, select Apply, Cancel, or Clear filters.

Use the Search box to filter the results by Username or Email address. Wildcards aren't supported.

If you select the Export report button, report generation progress is shown as a percentage of complete. In the dialog that opens, you can choose to open the .csv file, save the .csv file, and remember the selection.

Repeat offenders tab for the Attack simulation report

A repeat offender is a user who was compromised by consecutive simulations. The default number of consecutive simulations is two, but you can change the value on the Settings tab of Attack simulation training at https://security.microsoft.com/attacksimulator?viewid=setting. For more information, see Configure the repeat offender threshold.

On the Repeat offenders tab, the chart shows the number of Repeat offender users and Simulated users.

The Repeat offenders tab in the Attack simulation report in the Microsoft Defender portal.

If you hover over a data point in the chart, the actual values are shown.

The details table below the chart shows the following information. You can sort the information by clicking on an available column header. Select Customize columns to change the columns that are shown. By default, all available columns are selected.

  • User
  • Simulation types
  • Simulations
  • Email address
  • Last repeat count
  • Repeat offenses
  • Last simulation name
  • Last simulation result
  • Last training assigned
  • Last training status

Select Filter to filter the chart and details table by one or more simulation type values:

  • Credential Harvest
  • Malware Attachment
  • Link in Attachment
  • Link to Malware

When you're finished configuring the filters, select Apply, Cancel, or Clear filters.

Use the Search box to filter the results by any of the column values. Wildcards aren't supported.

Use the Export report button to save the information to a CSV file. The default filename is Attack simulation report - Microsoft Defender.csv, and the default location is the local Downloads folder. If an exported report already exists in that location, the filename is incremented (for example, Attack simulation report - Microsoft Defender (1).csv).

Simulation report in Attack simulation training

The simulation report shows the details of in-progress or completed simulations (the Status value is In progress or Completed). To view the simulation report, use any of the following methods:

The report page that opens contains Report, **Users, and Details tabs that contain information about the simulation. The rest of this section describes the insights and reports that are available on the Report tab.

The sections on the Report tab for a simulation are described in the following subsections.

For more information about the Users and Details tabs, see the following links.

Simulation report for simulations

This section describes the information in the simulation report for regular simulations (not Training campaigns).

The Report tab in the simulation report in Attack simulation training.

Simulation impact section in the report for simulations

The Simulation impact section on Report tab** for a simulation shows the number and percentage of Compromised users and Users who reported the message.

If you hover over a section in the chart, the actual numbers for each category are shown.

Select View compromised users to go to the Users tab tab in the report where the results are filtered by Compromised: Yes.

Select View users who reported to go to the Users tab tab in the report where the results are filtered by Reported message: Yes.

The Simulation impact section on the Report tab of a simulation report for a simulation.

All user activity section in the report for simulations

The All user activity section on Report tab** for a simulation shows numbers for the possible outcomes of the simulation. The information varies based on the simulation type. For example:

  • Clicked message link or Attachment link clicked or Attachment opened
  • Supplied credentials
  • Read message
  • Deleted message
  • Replied to message
  • Forwarded message
  • Out of office

Select View all users to go to the Users tab tab in the report where the results are unfiltered.

The All users activity section on the Report tab of a simulation report for a simulation.

Delivery status section in the report for simulations

The Delivery status section on Report tab** for a simulation shows the numbers for the possible delivery statuses for the simulation message. For example:

  • Successfully received message
  • Positive reinforcement message delivered
  • **Just simulation message delivered

Select View users to whom message delivery failed to go to the Users tab tab in the report where the results are filtered by Simulation message delivery: Failed to deliver.

Select View excluded users or groups to open an Excluded users or groups flyout that shows the users or groups that were excluded from the simulation.

The Delivery status section on the Report tab of a simulation report for a simulation.

Training completion section in the report for simulations

The Training completion section on the simulation details page shows the trainings that are required for the simulation, and how many users completed the trainings.

If no trainings were included in the simulation, the only value in this section is Trainings were not part of this simulation.

The Training completion section on the Report tab of a simulation report for a simulation.

First & average instance section in the report for simulations

The First & average instance section on Report tab** for a simulation shows information about the time it took to do specific actions in the simulation. For example:

  • First link clicked
  • Avg. link clicked
  • First credential entered
  • Avg. credential entered

The First & average instance section on the Report tab of a simulation report for a simulation.

Recommendations section in the report for simulations

The Recommendations section on Report tab** for a simulation shows recommendations for using Attack simulation training to help secure your organization.

The Recommendations section on the Report tab of a simulation report for a simulation.

Simulation report for Training campaigns

This section describes the information in the simulation report for Training campaigns (not simulations).

The Report tab in the Training campaign report in Attack simulation training.

Training completion classification section in the report for Training campaigns

The Training completion classification section on Report tab** for a Training campaign shows information about the completed Training modules in the Training campaign.

The Training completion classification section on the Report tab in the Training campaign report in Attack simulation training.

Training completion summary section in the report for Training campaigns

The Training completion summary section on Report tab** for a Training campaign uses bar graphs show the progression of assigned users through all Training modules in the campaign (number of users / total number of users):

  • Completed
  • In progress
  • Not started
  • Not completed
  • Previously assigned

You can hover over a section in the chart to see the actual percentage in each category.

The Training completion summary section on the Report tab in the Training campaign report in Attack simulation training.

All user activity section in the report for Training campaigns

The All user activity section on Report tab** for a Training campaign uses a bar graph to shows how main people Successfully received training notification (number of users / total number of users).

You can hover over a section in the chart to see the actual numbers in each category.

The All user activity section on the Report tab in the Training campaign report in Attack simulation training.

Get started using Attack simulation training

Create a phishing attack simulation

create a payload for training your people