Social Engineering Drills That Work: From Who Clicked to How Many Reported

Why companies do it: what Taiwan's rules require

Listed companies: the TWSE cybersecurity control guidelines

The Taiwan Stock Exchange's Cybersecurity Control Guidelines for Listed and OTC Companies (上市上櫃公司資通安全管控指引) require companies to run an email social engineering drill at regular intervals every year, give training to staff who opened the email or link by mistake, and keep records^[1]. The guidelines also require everyone who uses information systems to attend security awareness sessions every year, and managers and staff responsible for security to take professional training every year^[1]. The TWSE cybersecurity section updated the guidelines and the amendment comparison table in November 2026 (the table is dated 9 November 2026); the drill clause moved from Article 30 to Article 31 with no change to its content^[1]. Under the supplementary provisions, unless laws or TWSE and TPEx rules provide otherwise, listed and OTC companies may adopt the guidelines as appropriate to their industry, size and security risk^[1].

Government and specific non-government agencies: the Cyber Security Management Act

Article 10 of the Cyber Security Management Act requires government agencies and specific non-government agencies to take part in security drills planned by the competent authority, with the content of the drills set by that authority^[2]. Specific non-government agencies are critical infrastructure providers, government-owned enterprises, designated foundations, and enterprises, organisations or institutions controlled by the government^[2].

The Regulations Governing the Reporting, Response and Drills for Cyber Security Incidents (資通安全事件通報應變及演練辦法) were formerly titled the incident notification and response regulations (資通安全事件通報及應變辦法); on 5 January 2026 the title and full text were amended, effective on issuance^[3]. The current regulations require the agencies directly under the Office of the President, the National Security Council and the five Yuan to run drills for themselves and the government agencies under or supervised by them, and municipal and county (city) governments to do the same for themselves, their subordinate agencies and the township (city) offices within their area; the drills must include at least one social engineering drill every six months and one incident reporting and response drill every year, and the results must be sent to the competent authority within one month of completion^[4]. Specific non-government agencies must take part in drills planned or run by their central industry competent authority, which may include social engineering drills; where a drill may infringe the agency's rights or legitimate interests, its written consent is required first^[4]. If a specific non-government agency breaches the drill provisions of these regulations, the central industry competent authority orders it to correct the breach within a deadline, and if it fails to do so, can impose a fine of NT$100,000 to NT$5 million for each further failure^[2].

In the Regulations Governing the Classification of Cyber Security Responsibility Levels, as amended on 7 January 2026, Appendices 1 to 8 do not list social engineering drills as a separate required item; the people-related requirement is training hours, for example every general user and manager at every level from A to E must receive at least three hours of general security awareness training each year^[5]. The appendices also let the central industry competent authority set additional requirements for the specific non-government agencies it oversees^[5], so those organisations still need to check with their regulator. For an overview of the law, see Taiwan's Cyber Security Management Act; for incident reporting and response drills, see Tabletop Exercises.

The rules ask for drills, records and follow-up

What these texts require is running drills regularly, keeping records and training people who clicked; none of them sets a click rate that must be met^[1][4][5]. Compliance decides that you must run drills; whether they are useful depends on how you design and measure them. Even companies outside these rules may be asked for drill records during a customer audit or certification, which is another reason to make the drills worth doing.

What click rate alone gets wrong

Click rate reflects the email, not just the people

NIST notes that drill results are usually measured by click rate (the number of people who clicked a link or attachment out of everyone sent the email) and reporting rate (the number who reported it to the organisation out of everyone sent the email), but these give a single point of insight into how many people "fell" for the phish and do not reflect how hard each phishing email was to detect^[6]. In its 2020 news release introducing the Phish Scale, NIST also warned that a low click rate may simply mean the training emails were too easy, lacked context relevant to the user, or resembled a previous exercise; looking at the numbers alone can create a false sense of security^[7].

In an expert roundtable hosted by NIST's Human-Centered Cybersecurity team and published in 2026, researcher Grant Ho described his large-scale study: people who received training were only about 2% less likely to fall for phishing than people who received none, while the simulated lures in the study consistently got up to 10% to 30% of employees to fall for them, or at least click the link^[8]. He also joked that improving your click rate is easy: send a very hard phishing email in month one and a very easy one in month two, and the click rate "drops"^[8].

The NIST Phish Scale: rate the difficulty first, then compare results

The NIST Phish Scale is a method for drill implementers to rate how hard a given phishing email is for its recipients to detect, and NIST published a user guide for it (NIST TN 2276) in November 2023^[6]. It looks at two things: first, the cues in the email that a recipient could notice, such as errors, the sender's display name and address, link URLs, and urgent or threatening language, where fewer cues means harder detection; second, premise alignment, how well the email's scenario fits the recipients, scored on five elements of 0 to 8 points each: whether it mimics a workplace process, its relevance to the recipients' work, whether it aligns with current internal or external events, whether it raises concern about consequences of not clicking, and whether the recipients have had targeted training or warnings^[6]. Combining the two gives a detection difficulty rating such as least difficult, moderately difficult or very difficult^[6].

Once difficulty is rated, results become comparable: a high click rate on a very difficult email is to be expected, whereas many clicks on a "least difficult" email are what signal that this group needs adjusted or additional training^[6]. NIST notes in the guide that the method is free for academic use and that commercial use requires a licence arranged through NIST^[6].

Punishing clickers only stops people reporting

The UK National Cyber Security Centre (NCSC) phishing guidance is blunt: no training package, including phishing simulations, can teach users to spot every phishing attempt; since no one can be expected to spot them all, punishing people for clicking on emails you sent yourself starts to resemble entrapment, so always check with HR before running any simulation^[9]. The NCSC adds that blaming users for clicking does not work: people click for many reasons, from personality traits to simply being busy and stressed, and threatening punishment does not change those factors; simulations also erode trust between employees and the security team, and employees who are afraid for their jobs will not report mistakes^[9]. NIST SP 800-50 Rev. 1 (September 2024) likewise advises that drills should not be punitive and no employee should be called out for their response, and that staff should be told in advance that the organisation runs phishing exercises at random and that results will guide future learning^[10].

Metrics that matter: reporting, timing and follow-through

The NCSC points out that metrics express what an organisation values: if you appear to value the absence of reported problems, you encourage people to keep quiet; it recommends including successes in your metrics, so that as well as counting how many people clicked, you focus on how many reported^[9]. The behaviour change metric suggested in NIST SP 800-50 Rev. 1 is likewise based on the number of people who received a phishing test email and reported it appropriately^[10]. In practice, you can track:

• Report rate: the share of recipients who reported the drill email through the official channel. Count reports made after clicking as well, and track them separately; the NCSC stresses that users should feel able to come forward even after they have clicked^[9].
• Time to first report: how long it takes from delivery to the first report, and to most of the reports. In an NCSC case study, a financial-sector company of about 4,000 staff received 1,800 fake invoice emails carrying variants of Dridex malware, and 1,750 were stopped by email filtering; of the 50 that reached inboxes, 25 were reported in total (including some after clicking), and those reports were the company's first sign that the attack had got past its first layer of defence^[9]. Of the 14 emails clicked, 13 failed to launch the malware because devices were up to date, and the single installation was detected and blocked when it called home^[9].
• Credential submission and attachment execution: clicking a link and typing a password into a fake sign-in page carry very different risks. Count "clicked", "submitted credentials" and "opened the attachment and enabled macros" separately so you know which step most needs work.
• Repeat clickers: people who click several times in a row get one-to-one help and more support, not punishment. The NCSC recommends extra support for staff more likely to be targeted, such as customer-facing departments that receive many unsolicited emails, and people who can access sensitive information or manage financial assets or IT systems^[9]. An industry researcher in the NIST roundtable also pointed out that some people's job is to handle large volumes of external email full of links every day, and that rather than punishing them, you should strengthen the technical controls around those roles^[8].
• Whether IT and the SOC acted on reports: when a real suspicious email is reported, how long triage takes, whether the same email is found and removed from other mailboxes, whether the malicious URL is blocked, and whether the reporter hears back. The NCSC recommends quickly telling reporters what action was taken so they know their reports make a difference^[9].

More reports also mean more false alarms. The industry researcher in the NIST roundtable saw this as acceptable: better to over-report than to have staff decide "that's probably not phishing" and never tell the security team; he also noted that a reporting tool that is hard to use in itself makes reporting difficult^[8].

Designing the drill

Before you start: set the rules and involve HR and legal

The NCSC recommends checking with HR before running a drill^[9]; NIST recommends involving the legal team in designing and reviewing drills, to avoid using real brands or naming government agencies in the bait, which could lead to emails or calls to those organisations^[10]. In Taiwan, drill emails that impersonate couriers, banks or government agencies deserve particular care. Staff should also be told in advance that drills happen at random and that results are used only to plan training^[10]; the NCSC notes that the message "you won't get in trouble for reporting" needs buy-in from HR, support functions and senior management^[9].

Vary the difficulty and match themes to the company

Rate every drill email with the Phish Scale first and show the rating alongside the results in your report^[6]. Mix easy and hard: emails that are too easy only produce a false sense of security^[7], while an industry expert in the NIST roundtable argued that an email with every clue removed has no educational value^[8]. Pick themes from the Phish Scale's premise elements, matching recipients' everyday processes^[6], for example fake invoices and payment notices for finance, delivery notifications, HR or payroll announcements, password expiry or MFA prompts, and messages asking people to scan a QR code; for what these look like in real attacks, see Phishing and Business Email Compromise.

Some themes call for caution even though attackers use them. In the same roundtable, Ho said the leadership of the organisation he studied was very concerned about using COVID or "you're fired" as topics; another industry expert said some lures cross the line regardless, such as bonuses at year end and anything about dismissal, and that drill platforms' built-in templates may include such topics, so users should not assume the templates have been vetted^[8]. Ho also noted the trade-off: if the goal is to prepare staff for real attacks, attackers will not avoid a topic because it is upsetting^[8]. In a Taiwanese context, themes such as year-end bonuses, pay rises, redundancy lists or abnormal health-check results should at minimum be discussed with HR in advance, with a clear plan for how to explain the drill to staff afterwards.

The NCSC also suggests an alternative: ask staff to write their own phishing emails, which gives them a much richer view of the persuasion techniques attackers use; a friendly competition between colleagues can avoid the "security team versus staff" feeling that simulations can create^[9].

One-click reporting, education in the moment

Add a "report phishing" button to the email client^[10], and make sure the reporting process is clear, simple and quick^[9]. The page a clicker lands on should point out the clues in the specific email they received, not a generic awareness page. Ho's study found that people shown a static education page after clicking did not improve, while those who completed interactive questions tailored to the simulated email they received were 20% less likely to fail later simulations than people who did not do it, but most people simply closed the training page^[8]. So keep the in-the-moment explanation short, and follow up in courses or team meetings.

Coordinate with email filtering

A drill email has to arrive before it can test anyone. In Microsoft 365, for example, allow lists cannot be used to let through messages identified as malware or high-confidence phishing; email from a third-party simulation platform is instead registered through the advanced delivery policy by sending domain and sending IP, and when users report these simulation messages with the built-in Report button, no alert or investigation is triggered^[11]. Microsoft also warns against working around limitations by adding the IP addresses of your own gateway or other third-party sending services to the simulation settings, since that effectively lets any sender impersonating the domain bypass spam filtering^[11]. Whatever gateway you use, limit the exception to the simulation platform's sending sources and review the settings when the drill ends.

Beyond email: phone calls, the help desk and deepfakes

The practical exercises listed in NIST SP 800-50 Rev. 1 include simulated email phishing, SMS phishing (smishing) and voice phishing (vishing)^[10]. Phone calls deserve particular attention: a joint advisory on the Scattered Spider group from the FBI, CISA and agencies in several other countries reports that the attackers pose as company IT or help desk staff by phone or SMS to obtain employees' credentials or persuade them to share MFA one-time codes; they also pose as employees to convince the help desk to reset passwords and move MFA to a device they control, and send repeated MFA push prompts until employees press accept^[12].

That makes the help desk itself a drill target: have a manager who is in on it arrange a call from an "employee who lost their phone and urgently needs MFA reset", and check whether the help desk follows procedure by calling back the number on file, verifying identity and notifying the real employee after the reset. For designing the reset procedure, see the account recovery section of the MFA Rollout Guide; for deepfake voice and video impersonation of executives, call-back verification and code words, see Impersonation and Deepfake Fraud. A researcher in the NIST roundtable also argued that against vishing and deepfakes, it is important to build verification through a safe out-of-band channel into the process for sensitive actions^[8].

Drills cannot replace technical controls

The NCSC divides phishing defence into four layers: make it difficult for attackers to reach users (for example DMARC, SPF, DKIM and email filtering), help users identify and report phishing, limit the effects of phishing that succeeds (for example malware protection and MFA), and respond quickly to incidents; it also notes that training, particularly phishing simulations, is the layer that is often over-emphasised^[9]. Besides continued staff training against vishing and spear phishing, the joint advisory above recommends requiring phishing-resistant MFA for all services to the extent possible^[12]. Click rates will never reach zero, so push ahead with phishing-resistant MFA and DMARC at the same time; if someone does enter company credentials on a phishing page, follow Compromised Company Email Account.

Action checklist

• Confirm which rules apply: listed and OTC companies should check Article 31 of the TWSE guidelines, run an email social engineering drill every year and keep records; specific non-government agencies should confirm drill arrangements and required items with their central industry competent authority.
• Agree with HR and legal before the drill: no punishment, no public naming, no real brands or agencies as bait, and sensitive themes such as bonuses, redundancies or health discussed in advance.
• Deploy a one-click report button in the email client, with an automatic acknowledgement and human feedback after each report.
• Rate every drill email with the Phish Scale; report difficulty, click rate, credential submission rate, report rate and time to first report together.
• Send clickers to a short interactive explanation tied to the email they received; give repeat clickers one-to-one help and check whether their role needs extra technical controls.
• Let drill emails through using your email system's dedicated mechanism, allowing only the simulation platform's sending domains and IPs, and review the settings after the drill.
• Use each drill to check how real reports are handled: how long triage takes, whether similar emails are removed, and whether reporters hear back.
• Run a phone drill against the help desk at least once a year to test identity checks for password and MFA resets.
• Push phishing-resistant MFA and DMARC in parallel, and draw drill themes from real cases in phishing advisories; if you need outside help designing drills and materials, see security awareness training.