Book an intro call NL

Knowledge base › Phishing

How to run a phishing test: step-by-step plan and pitfalls

Last updated

A phishing test with your own employees is quickly thought up, but things also go wrong quickly: an annoyed works council, an IT department raising the alarm, or a report nobody can act on. This step-by-step plan helps you run a phishing test that actually delivers something, and points out the pitfalls at each step.

First: what do you want to know?

A phishing test is a measurement. So decide in advance which question you want to answer. Do you want a baseline for the whole organisation? Do you want to know whether the finance department recognises CEO fraud? Or do you want to show that a training course had an effect? The question determines the target group, the scenario and how you read the report.

Do not look only at who clicks. A good phishing test measures three things: who clicks, who then enters details and who reports the email. The report rate is the figure you want to see rise over time. More on what a test measures is on the page phishing test for employees.

Step-by-step plan: a phishing test in 8 steps

  1. Set the goal and target group. Write down in a few sentences what you want to measure, with whom and when. Include management in the target group: directors are a favourite target of CEO fraud.
  2. Arrange permission. Have the management or another authorised person give written approval. If your organisation has a works council, involve it beforehand: in the Netherlands, an arrangement suitable for observing employee behaviour requires its consent (Article 27(1)(l), Dutch Works Councils Act).
  3. Arrange privacy. The results are personal data. Record which data you keep, for how long and who sees it. If you work with an external party, sign a data processing agreement (Article 28 GDPR).
  4. Inform IT and the reporting point. Have your IT department or IT provider allowlist the sending addresses, so the emails arrive. Agree who knows the test is running, so that a report does not turn into a real incident response. Also make sure employees know where they can report a suspicious email.
  5. Choose a scenario. Pick an email that matches what your employees really receive: a parcel notification, a Microsoft 365 alert, an invoice. Do not start with the hardest scenario; a baseline has to be recognisable.
  6. Build a landing page with a learning moment. Anyone who clicks lands on a page explaining that it was a test and how the email could have been recognised. Short, friendly and without blame.
  7. Send gradually. Send the emails over 2 to 4 weeks, not all at once. Otherwise colleagues warn each other at the coffee machine and you mainly measure how fast a rumour spreads.
  8. Report and follow up. Produce a report per department with the click, submission and report rates, discuss it with management and give employees a follow-up, for example a short phishing training. Schedule the repeat straight away: only a second measurement shows whether anything has changed.

The pitfalls of a phishing test

  • Looking only at the click rate. A low click rate with nobody reporting is weaker than a slightly higher click rate with quick reports.
  • Naming people. Confronting employees about a click mainly teaches them to stop reporting. Report per department.
  • Not informing IT. The emails then get blocked, or a real incident response starts that costs time and trust.
  • Counting opens as a result. Email programs load images automatically or not at all. An open rate therefore says little.
  • Counting clicks by scanners. Security filters on mail servers sometimes open links automatically to check them. For strikingly fast clicks, right after sending, check whether a scanner was responsible.
  • Themes that go too far. A fake bonus, a dismissal notice or a message about a colleague's health works well, but damages trust. Choose scenarios you can explain afterwards without embarrassment.
  • Testing only once. A single measurement is a snapshot. Without a repeat you do not know whether the explanation and training are working.

Do it yourself or outsource?

You can run a phishing test yourself, if you have time for the technology (sending, a landing page, measuring), the legal preparation and the report. If you would rather not do that yourself, you can outsource it. Lumyo handles the entire phishing simulation: permission, scenarios, delivery, report and follow-up, at fixed prices. What that costs is explained under security awareness training costs.

Does your organisation fall under NIS2? Then members of the management body must take cybersecurity training themselves. Read what that means in NIS2 and the training obligation for management.

Frequently asked questions about running a phishing test

Does the works council have a say in a phishing test?

Often, yes. A phishing test records the behaviour of employees. In the Netherlands, an arrangement suitable for observing or monitoring the behaviour or performance of employees requires the consent of the works council (Article 27(1)(l) of the Dutch Works Councils Act). If your organisation has a works council, involve it beforehand.

May I share the names of employees who click with their manager?

We advise against it. The results are personal data, so the GDPR asks for as little identifiable data as possible. Report per department and only give identifiable results to the contact person you appoint in advance.

How long does a phishing test take?

Allow 2 to 4 weeks in which the emails are sent out gradually, plus a few weeks of preparation for permission, the scenario and coordination with IT.