CONTENTS
- 1. AI Security | Generative AI, the Most Common Yet Invisible Channel for Data and Technology Leaks
- - Changes in the Methods of Leakage
- 2. AI Security | A Threat That Shifts from a Technical Problem to a Management Risk
- - Input-Based Disclosure of Confidential Information
- - Prompt Injection and Adversarial Attacks
- - Ambiguity in the Structure of Liability
- 3. AI Security | The Core of the AI Data Governance That Companies Must Build
- - A Structure in Which Data Does Not Leave
- - Technical Controls Blocked at the Input Stage
- - Management Based on the AI Lifecycle
- - Clarification of the Party Responsible
- 4. AI Security | An Era in Which the Level of Security Determines Corporate Trust and Investment Evaluation
- - AI Is Not Something to Block but a Management Asset to Be Managed
1. AI Security | Generative AI, the Most Common Yet Invisible Channel for Data and Technology Leaks
The reason AI security has become important is clear.
Generative AI has already become the most frequently used everyday productivity tool in the workplace, yet at the same time it is rapidly establishing itself as the data and technology leak channel that companies are least able to control.
The problem is that many companies still perceive AI as nothing more than a work tool.
In practice, however, AI is functioning as new leak infrastructure that follows email, USB drives, and external hard drives, and because it is harder to detect and control than conventional leak methods, it significantly raises the difficulty of a company's response.
According to recent analyses of overseas corporate environments, a considerable number of employees are confirmed to be already using generative AI tools.
The problem is that many of them are using AI through personal accounts that fall outside the company's scope of management and control.
Changes in the Methods of Leakage
The fact that technology leaks are occurring in ways that are difficult to capture with existing security systems is also a point that raises the difficulty of response.
Existing security systems are designed around file uploads or storage actions, but a considerable portion of actual data and technology leaks occurs not through files but through copying and pasting, that is, during the prompt input process.
∙ A considerable portion of this use is based on personal accounts
∙ Leakage occurs through the clipboard, a fileless channel
∙ Detection itself is difficult with existing DLP and security logs
As a result, a company is placed in a position where it not only fails to prevent leaks but also fails even to recognize that a leak has occurred.
In the process of using AI, therefore, a workflow with weakened visibility and control acts as a greater risk factor than the technology itself.
In overseas corporate environments, there have indeed been reported cases in which the structure of a core algorithm was exposed externally through prompt input alone during the review of development code or the organization of technical documents.
2. AI Security | A Threat That Shifts from a Technical Problem to a Management Risk
AI security issues have now entered a stage where data and technology leaks caused by generative AI must be recognized not as incidents but as a structural management risk.
The representative types of threats are as follows.
Input-Based Disclosure of Confidential Information
The moment an employee enters confidential information as a prompt, that information can leave the company's domain of control.
Depending on how the AI service operates, it is difficult to completely rule out the possibility that the input information may be used as log or analytics data.
Prompt Injection and Adversarial Attacks
This is an attack method that, through external input, induces the AI to expose unintended data or to bypass internal rules.
The risk grows particularly in automated tasks such as customer response, document summarization, and code verification.
This can lead beyond a minor security incident to a risk in which a company bears unintended legal and contractual liability in the course of automated decision-making.
Ambiguity in the Structure of Liability
Because AI outputs appear in the form of secondary generated works, it is very difficult to subsequently prove their identity with the original technology, the leak channel, and the party responsible.
Accordingly, a company may face the following management risks.
∙ Being assessed as a company with inadequate security management during investment and collaboration
∙ Risk of violating regulations related to personal information and industrial technology
∙ A structural vulnerability that fails to prevent recurrence even after an incident
AI security is now expanding beyond the domain of the information security department or a particular officer into a management risk that must be managed across the entire organization.
3. AI Security | The Core of the AI Data Governance That Companies Must Build
For the sake of AI security, some companies choose to ban the use of generative AI outright, but this is unlikely to be a realistic alternative.
In an organization that has experienced the efficiency and speed of its work, the moment AI use is completely blocked, AI moves out of official systems and into Shadow AI.
Here, Shadow AI refers to the practice in which employees use personal accounts or external AI services without the company's approval or management.
In this case, recording and monitoring of the input and output data become impossible, and it also becomes difficult to confirm afterward what information was provided externally and through what channel.
In particular, as sensitive data such as code, technical documents, and customer information is entered as prompts without separate controls, the problem arises that security policies and internal control systems are effectively neutralized.
Control disappears, and instead the risk expands, giving rise to a secondary issue.
What is needed, therefore, is not a ban but a controllable structure of use, that is, data governance.
What Is Data Governance?
In an AI environment, an integrated control structure that also encompasses the scope of AI input and output data, the conditions of permissible use, and the methods of recording and monitoring becomes the core of data governance.
To prevent leaks through AI, one must first identify what is subject to protection.
Without defining the scope of core technologies and data, any control can only remain a formality.
Without this premise in place, no security system or policy can readily be effective.
A Structure in Which Data Does Not Leave
When a public generative AI is used as is, the content entered by employees is transmitted to the servers of an external service rather than staying inside the company.
In this process, it is difficult for a company to directly confirm or control where that data is stored, how it is processed, and whether it is again used for training.
By contrast, in an on-premises environment where a company maintains its own servers and operates AI only within internal systems, or in a Private AI structure based on a dedicated VPC where, even when an external cloud is used, AI is operated in an independent space separated for the company alone, a company can directly manage the scope of data storage, processing, and access.
In that such a structure can control data so that it does not leak externally, it is now establishing itself not as an option but as a basic premise.
Technical Controls Blocked at the Input Stage
Policies or pledges alone have their limits.
To prevent mistakes, a physical braking mechanism beyond discipline is needed.
∙ Masking of personal information and core technology, or restrictions on their input
∙ Control over the act of copying and pasting itself
Management Based on the AI Lifecycle
Controls must operate across all stages of data collection, training, inference, and disposal.
∙ An unlearning structure that can remove only specific data when necessary
∙ Differentiated access permissions by role and their automatic revocation
Clarification of the Party Responsible
Technology and data leaks caused by generative AI are now an area that calls for a clear assignment of responsibility.
There is a need to designate the parties responsible for AI use policies, access controls, and incident response, and to incorporate AI items into existing security frameworks such as ISMS-P.
This is also directly connected to a legal defense structure for proving that a company fulfilled its management responsibilities when an incident occurs.
4. AI Security | An Era in Which the Level of Security Determines Corporate Trust and Investment Evaluation
Just as technology leaks have spread into a risk for entire industries, the level of AI security has now begun to operate as a factor in evaluating a company, beyond an individual incident.
▶ Review of the level of AI use control during investment due diligence
▶ Strengthened security requirements in the course of overseas collaboration and data transfer
▶ Shifting into a question of management responsibility when regulatory compliance fails
How a company sets and manages the scope of AI use and the parties responsible is important.
AI Is Not Something to Block but a Management Asset to Be Managed
AI is a tool that enhances a company's competitiveness, and at the same time, if left uncontrolled, it can become the fastest channel for information leakage.
Accordingly, it is important to review the organization's decision-making structure and accountability framework together.
To respond to such changes, Daeryun Law Firm LLP launched its AI and Data Intelligence Group and built a data-centered system for case analysis and risk response that links AI compliance, industry-specific AI strategy, cybersecurity and crisis response, and digital forensics and e-discovery.
If you need a review of your AI use structure, a diagnosis of data leak risks, or the design of internal AI governance, you are welcome to review your response strategy through an integrated diagnosis from the perspectives of security solutions and management risk.
Related News
