Most companies' employees already use tools such as ChatGPT, Copilot and Gemini to draft email, translate, summarize meetings and fix code. The usual next step is to connect AI to company data and systems: letting an assistant read mailboxes and shared folders, browse the web, call internal APIs or even run code. The risk grows with it, from "an employee pasted something they shouldn't have" to "the AI read instructions an attacker planted and carried them out with the permissions the company gave it".

This article is for managers and IT staff whose employees already use generative AI and who are about to connect AI assistants or agents to company data. It covers data leakage and shadow AI, prompt injection and AI agent permissions, how attackers use AI, the NIST frameworks and the state of Taiwan's rules, and ends with an outline for a company AI use policy.

Employees pasting data into AI: shadow AI and terms of service

In May 2023 Samsung Electronics confirmed to CNBC that, after discovering cases of misuse, it was temporarily restricting the use of generative AI on company computers; Bloomberg reported the same day that some staff had uploaded sensitive code to ChatGPT[1]. Samsung also advised employees not to enter any personal or company-related information into such services when using them outside work[1]. Incidents like this are rarely deliberate leaks: employees want to work faster, and the company has neither provided approved tools nor said clearly what must not be pasted. Employees' use of AI services the company has not approved is commonly called "shadow AI".

What happens to data once it is pasted depends on the type of account and which terms apply. Taking Google's consumer Gemini as an example, Google says a subset of chats is reviewed by human reviewers (including trained reviewers from its service providers), and explicitly asks users not to enter confidential information they would not want a reviewer to see or Google to use to improve its services[2]. With the Keep Activity setting off and no feedback submitted, future chats are not used to improve Google's AI models, but they are still kept with the account for 72 hours; chats that have been reviewed are kept for up to three years and are not deleted when the user deletes their activity[2]. The same page notes that use with a work or school account may be subject to different data handling terms[2].

Enterprise services come with different terms. Microsoft, for example, says that prompts, responses and data accessed through Microsoft Graph in its enterprise Copilot are not used to train foundation models, and are processed and stored under the same contractual commitments as the organization's other Microsoft 365 content[3]. But the enterprise version raises a different problem: Copilot only surfaces organizational data the user has at least view permission for, and Microsoft therefore stresses using the permission models in services such as SharePoint so that the right people have access to the right content[3]. Put another way, whatever a user can see, the AI can find for them: a payroll sheet or acquisition file that was "shared with everyone" but never found may turn up in answer to a single question once an AI assistant is rolled out. Tighten sharing settings first; see Microsoft 365 and Google Workspace security baselines.

OWASP's Top 10 for LLM Applications (2025 edition) lists Sensitive Information Disclosure as LLM02, covering personal data, financial details, health records, confidential business data, security credentials and legal documents. It warns users that sensitive data they provide unintentionally may later be disclosed in the model's output, and recommends that providers' terms of use let users opt out of having their data used for training[4]. This is a separate list from the general OWASP Top 10 for web applications, which is covered in what the OWASP Top 10 is. The practical conclusion for companies: rather than banning AI outright and pushing employees onto personal accounts, provide approved enterprise tools and state clearly which data must never be entered (see the policy outline below).

Prompt injection and AI agents: what the AI reads may be an attacker's instructions

Direct and indirect prompt injection

Prompt injection is number one (LLM01) in the 2025 edition of OWASP's Top 10 for LLM Applications. It occurs when input alters the model's behavior or output in unintended ways, and the input does not need to be visible to humans as long as the model parses it[5]. OWASP distinguishes two kinds[5]:

  • Direct prompt injection: the user's own input changes the model's behavior, whether as a deliberate attack or by accident. For example, an attacker tells a customer support chatbot to ignore its guidelines, query private data stores and send emails.
  • Indirect prompt injection: content the model reads from external sources such as websites or files carries instructions. For example, a user asks an AI to summarize a web page containing hidden instructions that make the AI insert an image linking to an external URL, exfiltrating the conversation; or an attacker modifies a document in a retrieval-augmented generation (RAG) knowledge base so that the AI gives misleading answers.

NIST describes indirect prompt injection as adversaries exploiting LLM-integrated applications remotely, without a direct interface, by injecting prompts into data likely to be retrieved; security researchers have already demonstrated using it to steal proprietary data or run malicious code remotely[6]. For a company, every external email, web page and customer-uploaded file an AI assistant reads is a possible channel for an attacker's instructions.

Worse, there is currently no complete fix. OWASP states plainly that, given the stochastic nature of how models work, it is unclear whether fool-proof prevention exists; what can be done is to mitigate the impact[5]. Vendor protections are also limited in scope: Microsoft says Copilot uses classifiers to detect jailbreaks and cross-prompt injection attacks (XPIA), but those classifiers may not be available in every Copilot scenario[3]. Design on the assumption that an injection will eventually succeed, and focus on what the AI can do once it has.

AI agents: permissions decide how much damage is done

When an AI can only chat, prompt injection at worst makes it say something wrong. Once it can read and send email, browse the web, call APIs or run code, for example through plugins or MCP (Model Context Protocol) servers connected to other tools, injected instructions turn into real actions. OWASP lists this as LLM06 Excessive Agency: the AI takes damaging actions because of hallucination, direct or indirect prompt injection, or a compromised extension or peer agent, and the root cause is usually excessive functionality, excessive permissions or excessive autonomy[7].

OWASP's example is close to what companies are deploying now: an AI assistant that only needs to summarize incoming email uses a plugin that can also send messages, and a malicious email uses indirect prompt injection to make the AI search the mailbox for sensitive information and forward it to the attacker[7]. An extension that can only read mail, an OAuth connection with a read-only scope, or requiring the user to press send on every message would each have prevented it[7].

This is not theoretical. Google Threat Intelligence Group (GTIG) reported in September 2026 that the financially motivated actor UNC6780 compromised legitimate developer accounts to publish trojanized forks of MCP servers to PyPI, and that its DUSTMAKER credential stealer drops malicious configuration files into hidden project directories read by AI coding assistants, instructing the assistant to run arbitrary commands during routine developer interactions[8]. AI tools, plugins and MCP servers are part of the software supply chain; for how to assess them, see supply chain attacks.

Reducing agent risk

  • Least privilege: offer only the tools and functions the task needs, and avoid open-ended tools such as "run any shell command" or "fetch any URL"; the identity a tool uses to reach downstream systems should have only the permissions it needs, for example read access to one table[7]. When an agent acts for a user, it should run in that user's context with the minimum OAuth scope, not through a shared high-privilege account[7].
  • Human approval for high-impact actions: sending email, payments, deletions, public posting and changes to production should be confirmed by a person before they happen[5][7].
  • Let systems, not the AI, decide authorization: implement authorization checks in downstream systems and validate every request made through an extension against security policy, instead of letting the model decide whether an action is allowed[7].
  • Separate untrusted content: keep external web pages, email and uploaded files apart from the user's instructions and label their source clearly; have the application handle API tokens in code rather than giving them to the model[5].
  • Logging, monitoring and rate limits: log the activity of extensions and downstream systems so unwanted actions are spotted early, and rate-limit actions such as sending email so monitoring can catch a problem before it grows[7].
  • Test the model as an untrusted user: run regular penetration tests and attack simulations to check that trust boundaries and access controls hold[5].

How attackers use AI

GTIG's September 2026 report says state-sponsored and cyber crime groups continue to use AI as a force multiplier across the attack lifecycle, from target reconnaissance and social engineering lures to malware obfuscation and post-exploitation troubleshooting[8]. In one example, a China-nexus espionage group with a history of targeting government entities used a tool to switch rapidly between models such as Claude, Gemini and Codex to write exploit scripts, generate convincing spear-phishing lures and debug errors[8]. The report also notes that attackers use models to speed up turning public disclosures into working n-day exploit code[8], which makes patching speed more important; see patch management for prioritization.

Automation is increasing too. GTIG observed a financially motivated actor that compromised an organization's cloud environment, deployed a multi-agent attack framework inside it, and planned, built and ran a mass credential harvesting campaign in under six hours, compromising thousands of third-party credentials[8]. The report also states, however, that GTIG has not yet seen threat actors deploy fully autonomous attack pipelines against targets in the wild, and that in information operations generative AI has delivered productivity gains rather than breakthrough capabilities[8]. NIST's generative AI profile likewise frames the information security risk as two-sided: AI lowers the barrier to hacking, malware and phishing, while AI systems themselves expand the attack surface[6].

The practical impact on companies comes down to three points:

  • More convincing fraud: NIST notes that multimodal models can produce highly realistic deepfake audio and video and can help attackers create fraudulent content that impersonates others[6]. Spotting typos or visual glitches is no longer enough; see phishing and business email compromise and impersonation and deepfake fraud.
  • AI assets are targets in their own right: GTIG recorded infostealers going after AI coding assistant configuration files that store API keys in plaintext, and an attacker who used a leaked GitHub personal access token to enter a cloud environment and run unauthorized AI workloads on the victim's compute[8]. For handling leaked credentials, see infostealers and leaked credentials.
  • The basics still work: the starting points in these cases include compromised developer accounts, leaked access tokens and breached cloud environments[8], so MFA, key and token management, and patching remain the first line of defense.

Frameworks and Taiwan's rules

NIST AI RMF and the Generative AI Profile

NIST released the AI Risk Management Framework (AI RMF 1.0) on 26 January 2023 for voluntary use, followed on 26 July 2024 by the Generative AI Profile (NIST AI 600-1)[9]. NIST's site now notes that AI RMF 1.0 is being revised as part of the White House AI Action Plan, and that on 7 April 2026 NIST released a concept note for an AI RMF Profile on Trustworthy AI in Critical Infrastructure[9].

AI 600-1 lists 12 risks unique to or exacerbated by generative AI. Those most relevant to corporate security are information security, data privacy, confabulation (often called hallucination), human-AI configuration (for example over-reliance on AI), information integrity, and value chain and component integration (non-transparent or untraceable third-party components and data)[6]. The 12 risks work well as a checklist for a pre-deployment risk assessment.

Taiwan: government agencies, financial institutions and listed companies

On 31 August 2023 the National Science and Technology Council (NSTC) presented to the Executive Yuan Council its draft Reference Guidelines for the Use of Generative AI by the Executive Yuan and Its Subordinate Agencies, which has ten points[10]. Key points: a human case officer must make the final, objective and professional judgment on generative AI output; generative AI must not be used to prepare classified documents; information that must be kept confidential, personal information and information not approved for disclosure must not be given to generative AI, and closed on-premises models may be used by classification level only after the system environment's security has been confirmed; unverified output must not be used directly as the basis for an administrative act or as the sole basis for a decision; use should be appropriately disclosed; and contractors must be required to observe the guidelines in procurement[10]. They apply to the Executive Yuan and its agencies, and state-owned enterprises, public schools, administrative corporations and government-funded foundations may also apply them[10]. Private companies are not bound, but the ten points make a good skeleton for a company policy.

The Artificial Intelligence Basic Act was promulgated on 14 January 2026 and took effect on promulgation. It has 20 articles, and the central competent authority is the National Science and Technology Council[11]. Its provisions mainly set out the government's principles and division of responsibilities and contain no penalties. On security, Article 4 lists "cybersecurity and safety" among the principles the government must follow in promoting AI, requiring security protections throughout AI research, development and use[11]. Article 16 requires the Ministry of Digital Affairs (MODA) to promote an AI risk classification framework, on which the competent authorities for each sector are to base risk-based management rules; Article 18 requires the government to finish enacting, amending or repealing the relevant regulations within two years of the Act taking effect[11]. MODA published version 1.0 of its AI Risk Classification Framework on 7 July 2026; it is addressed to the sector competent authorities, and it suggests industry start with its table of AI risk types[12]. Concrete management rules for each sector are to be issued separately by the competent authorities under the framework.

For the financial sector, the Financial Supervisory Commission (FSC) issued its Guidelines for the Use of Artificial Intelligence in the Financial Industry on 20 June 2024 as administrative guidance[13]. They suggest that, for aspects a financial institution has little control over, it should set out the division of risk-monitoring responsibilities with its vendors by contract, and that where it uses third-party generative AI without insight into how it was trained, its own staff must still apply objective and professional control to the output[13]. On 13 October 2026 the FSC also issued a consultation paper on the security of agentic AI in the financial industry for 30 days of comment. It proposes managing by "use case", clarifying the data an agent may access, the tools it may call, the actions it may take and where its output goes; grading use cases on three axes of data sensitivity, action impact and capability boundaries; and treating agentic AI as a non-human identity and workload with the capacity to act, controlled under the zero trust principles of least privilege and traceability[14]. It is only a consultation paper so far, and the FSC says it does not replace existing laws or internal control requirements[14], but its approach is worth borrowing outside finance too.

Listed companies should note the Taiwan Stock Exchange's November 2026 revision of the Cybersecurity Control Guidelines for Listed and OTC Companies: the table of amendments is dated 9 November 2026, and Articles 38 and 39 are new[15]. Article 38 provides that where the use of AI systems or services involves core business or sensitive data, or can interface with, operate, modify or trigger functions of information systems, the company shall adopt the following measures according to the nature of the business and the level of risk[16]:

  1. Consider adopting MODA's AI Risk Classification Framework or AI guidelines issued by the competent authority for the company's sector.
  2. Preferably establish security rules for employees' use of AI systems or services, and take appropriate measures against unauthorized use.
  3. When introducing, developing or significantly changing AI systems, carry out security assessment or testing appropriate to the risk, review model operation regularly, and manage supply chain risk for AI models and third-party components.
  4. When using agentic AI, restrict its purposes, data, tools, destinations and execution permissions; for automated operations with a major or irreversible impact on core business, sensitive data or information systems, put human approval, security guardrails or other controls in place.
  5. Establish monitoring and logging appropriate to the risk, so that important operations, behavioral deviations and anomalies can be traced.

Article 39 provides that AI used only for low-risk support such as lookup, summarization, translation, training or internal drafts, and not falling under the previous article, can be handled under existing security measures[16]. The reference documents listed in the table of amendments include MODA's framework, the two FSC documents, the Ministry of Economic Affairs Industrial Development Administration's guide to AI adoption in manufacturing, ISO/IEC 42001:2023, and NIST AI RMF and AI 600-1[15]. Article 44 lets companies adopt the guidelines as appropriate to their industry, size and security risk[16]; for changes to the other chapters, see the TWSE cybersecurity control guidelines.

Outline of a company AI use policy

Whether or not any of these rules bind your company, it should have a written AI use policy. The outline below draws on the Executive Yuan guidelines, OWASP and the new TWSE articles; scale it to the size of your company:

  1. Scope and approved tools: list the approved AI services, editions and account types, and require sign-in with company accounts. Personal accounts come with different data handling terms from enterprise editions[2][3] and must not be used for company data. New tools, plugins and MCP servers need a security review before use.
  2. Data that must never be entered: for example classified documents and trade secrets, personal data, customer data, passwords and API keys, and unpublished financial information and material information. The Executive Yuan guidelines' bans on official secrets and personal information can be borrowed directly[10]; reviewed enterprise or on-premises tools can then be opened up step by step according to data classification.
  3. Output review: users bear final responsibility for output, must not use it unverified as the sole basis for a decision, and should disclose AI use where appropriate[10]. External documents are reviewed by a person before they go out, and AI-generated code goes through the same code review and security testing as any other code.
  4. Permissions for AI agents and plugins: register every agent, plugin or MCP server with its purpose, accessible data, callable tools, permissions and owner, under least privilege; sending email, payments, deletions and production changes require human approval[7][16].
  5. Logging and monitoring: keep records of prompts, responses and tool calls so that important operations and anomalies can be traced[16]. At Microsoft, for example, admins can review this data with Content search or Microsoft Purview and set retention policies for Copilot interactions[3]. For setting retention periods, see what to log and how long to keep it.
  6. Incident reporting: employees need to know whom to tell, and how quickly, when they have pasted secrets or personal data into an unapproved AI service or an AI agent has done something unexpected; where personal data is involved, follow the personal data breach response procedure.
  7. Procurement and outsourcing: require vendors to follow the company's AI use rules, as point 8 of the Executive Yuan guidelines also does[10], and set out by contract the division of responsibility for data use, retention and risk monitoring of AI services[13].
  8. Training and regular review: cover shadow AI, prompt injection and AI-generated phishing in annual security awareness training, and review the approved list and the policy at least once a year.

Action checklist

  • Inventory the AI services and browser extensions employees actually use, separating company-approved from personal accounts.
  • Provide at least one approved enterprise AI tool, confirm its contract does not allow training on company data, and centralize sign-in through company accounts.
  • Publish an AI use policy that lists the data categories that must never be entered, and include it in onboarding and annual awareness training.
  • Before rolling out an enterprise AI assistant, review company-wide and external sharing in SharePoint, OneDrive, Google Drive and Teams.
  • Keep a register of every AI agent, plugin and MCP server: purpose, data, tools, permissions and owner; remove anything unused.
  • Require human approval for sending email, payments, deletions and production changes, and give agents only read-only or minimum-scope authorization.
  • Turn on and retain audit logs for AI services and feed them into existing monitoring and alerting.
  • Store AI service API keys centrally and rotate them regularly; keep them out of code and plaintext configuration files.
  • Listed companies: map AI uses against Articles 38 and 39 of the TWSE guidelines, separate high-risk from low-risk uses, and report to management.
  • Test internal AI applications for prompt injection; for outside testing, see our vulnerability assessment service.