“The Illusion of Performance Secured Through Distillation”: Kimi K3 Exposes Cyberattack Limitations, Putting China’s Core AI Capabilities to the Test

Kimi K3 demonstrates weak cyberattack capabilities in a joint US-UK assessment
Distillation-based training fails to produce advanced offensive hacking competence
China faces growing pressure to prove original capabilities after repeated disputes with the US over model distillation

A joint study by the British and US governments has found that Kimi K3, developed by China’s Moonshot AI, falls significantly behind leading American artificial intelligence models in its ability to conduct cyberattacks. Kimi K3 had recently emerged as a symbol of the rapid technological advance of Chinese open-source AI after approaching proprietary models in security-vulnerability detection, but it displayed clear weaknesses in offensive tasks requiring complex judgment, planning, and execution. Market observers argue that the gap may reflect the limitations of model distillation, in which a system is trained extensively on outputs produced by more advanced American models. Distillation can reduce development time and cost while narrowing superficial performance gaps, but it does not necessarily allow a model to acquire original capabilities for solving difficult problems that were absent from its training data.

Performance Weaknesses in Kimi K3

On July 24, local time, the UK AI Security Institute under the Department for Science, Innovation and Technology and the US Center for AI Standards and Innovation under the National Institute of Standards and Technology released a joint report evaluating Kimi K3’s cybersecurity performance. The researchers used ExploitBench, a public benchmark designed to measure how effectively an AI model can analyze software vulnerabilities and develop attack code capable of exploiting them. Kimi K3 recorded an overall ExploitBench score of 32.2%, far below the average score of 76.2% achieved by the leading US models evaluated alongside it.

The most pronounced gap appeared in arbitrary code execution, the highest-level attack category in which an attacker obtains the ability to execute commands and potentially take complete control of a system. Kimi K3 failed to achieve arbitrary code execution in any of the benchmark’s 41 tasks, while leading American models succeeded in 20. A similar result emerged in The Last Ones, a cyber-range test simulating an enterprise network. In a 32-stage attack path containing four subnets and more than 20 hosts, advanced US models reached an average of 28.5 stages, whereas Kimi K3 progressed only to an average of 17. The report concluded that although Kimi K3 is regarded as one of the strongest large language models currently available in China, its offensive cyber capabilities remain substantially below those of the latest frontier systems.

Findings Contrast Sharply with Earlier Analysis

The results stand in clear contrast to a recent assessment by Swiss security company Aikido Security. In research published on July 16, Aikido found that Kimi K3 delivered the strongest performance among open-source models in detecting recently discovered security vulnerabilities. The company compared the bug-detection ability and cost efficiency of major AI models using 26 newly identified vulnerabilities that had not yet been publicly disclosed.

Kimi K3 detected 23 of the 26 vulnerabilities, matching the performance of OpenAI’s mid-tier GPT-5.6 Terra model. Although it performed slightly below the premium GPT-5.6 Sol system, its operating cost was estimated at only around one-quarter of Sol’s. Aikido researcher Philippe Durasov explained that the assessment was based on recently discovered, nonpublic vulnerabilities, making it highly unlikely that Kimi K3 had previously encountered the test data during training. He argued that the findings demonstrated a substantial improvement in the Kimi model family and suggested that open-source systems could no longer automatically be considered inferior to proprietary frontier models.

Structural Limits of Distillation Become Visible

The contradictory assessments appear to reflect differences in both the nature of cybersecurity work and the capabilities measured by each test. Aikido Security focused on the passive identification of software weaknesses and the analysis of defensive security issues within source code. ExploitBench and The Last Ones, by contrast, measured active offensive capabilities, including exploiting software flaws, obtaining control of a system, and completing a multistage network intrusion. The evaluations were therefore designed to test fundamentally different forms of competence.

Experts believe the resulting performance gap may arise from structural limitations inherent in distillation-based training. Kimi K3 is widely believed to have improved its performance by collecting and learning from large volumes of output generated by the most advanced systems developed by Anthropic, OpenAI, and other leading US AI companies. The problem is that these source models operate under sophisticated safety policies designed to refuse requests for detailed cyberattack code, advanced exploit generation, and other dangerous material. The datasets used to train Kimi K3 may therefore have contained little meaningful information about offensive algorithms, exploit construction, or real-world attack procedures. Such gaps in the training data would translate directly into an absence of deep hacking capabilities.

Distillation allows a smaller or less advanced system to imitate patterns found in the answers of a stronger model, but its effectiveness depends heavily on the information contained in those answers. If the teacher model refuses to reveal certain reasoning processes or technical procedures, the student model cannot easily acquire them through imitation alone. Kimi K3 may consequently reproduce the surface-level analytical style and defensive coding knowledge of frontier models while lacking the internal problem-solving structures required to design and execute novel attacks in unfamiliar environments.

US and China Clash Over Adversarial Distillation

Market observers expect the debate surrounding Kimi K3 to renew criticism of Chinese companies’ extensive use of model distillation. Distillation itself is a common technique throughout the AI industry, but US officials increasingly characterize China’s large-scale use of outputs from proprietary American models as both an intellectual-property and national-security concern. American AI companies have made similar accusations, arguing that some Chinese firms are not conducting ordinary research-oriented distillation but engaging in “adversarial distillation,” in which large numbers of false accounts and proxy networks are used to collect commercial AI outputs without authorization.

Anthropic recently submitted a formal complaint to the US Senate and Department of Defense alleging that an AI research organization affiliated with Alibaba had conducted a large-scale unauthorized knowledge-distillation attack against Claude, its latest AI model. According to the complaint, the organization used approximately 25,000 false accounts and automated bot programs to conduct 28.8 million conversations with Claude over only 44 days, effectively attempting to extract the model’s accumulated knowledge and behavior. Moonshot AI, DeepSeek, and MiniMax were also reported to have used approximately 24,000 false accounts to exchange more than 16 million questions and answers with Claude. The information collected reportedly concentrated on advanced capabilities including agentic reasoning, coding, tool use, and computer control. OpenAI has similarly warned in submissions to the US Congress that Chinese AI companies continue to attempt unauthorized distillation of its models.

Market Evaluation Standards Begin to Change

Chinese companies’ distillation methods are also becoming increasingly sophisticated. Earlier approaches resembled copying an answer sheet, but more recent techniques attempt to extract the multistage logical structure used to derive an answer and inject that process into a separate model. Chinese developers are also believed to operate automated collection systems that repeatedly disguise internet protocol addresses as locations in the United States or Europe to circumvent security controls imposed by American providers. Some firms then release models created through the distillation of proprietary US systems as free open-source products worldwide, effectively re-exporting capabilities originally developed behind closed platforms.

These practices have generated persistent concern that the existing six- to nine-month technological gap between the United States and China could close rapidly if such activity remains unchecked. The apparent fundamental limitations of Kimi K3, however, may cause market perceptions of distillation to change direction. Until now, distilled models have primarily been judged according to how closely they could reproduce the benchmark performance of established systems within a short period and at a low cost. Future evaluations may place greater emphasis on whether a model possesses “original intelligence,” meaning the ability to solve problems absent from its training data and adapt successfully to unfamiliar environments.

“With US opposition to unauthorized distillation intensifying, Kimi K3 has become evidence that imitation alone does not necessarily produce independent technological capability,” one AI industry participant said. “Chinese AI companies will increasingly face pressure to demonstrate not merely that they can catch up in benchmark performance, but that they have accumulated their own data, training systems, research methods, and foundational technologies.”