Malicious AI Models Found on Hugging Face

What is Hugging Face?

Hugging Face is one of the most widely used platforms for sharing and collaborating on machine learning (ML) models, datasets, and AI applications. Originally known for its advancements in natural language processing (NLP), Hugging Face has since expanded to host a vast collection of models spanning multiple AI domains, including computer vision, speech recognition, and reinforcement learning.

The platform serves as a hub for researchers, developers, and organizations, providing them with tools to easily share and deploy machine learning models. Hugging Face promotes an open-source philosophy, making cutting-edge AI models more accessible to a broader audience while fostering collaboration in the AI community.

However, with great accessibility comes security risks. Recent reports have revealed that cybercriminals are exploiting the platform to distribute malicious AI models, bypassing existing security measures and infecting unsuspecting users.

Why Are So Many AI Models Needed?

AI models serve a variety of specialized tasks, and no single model can excel at everything. Different domains require different architectures, training methodologies, and optimizations. Below are a few key areas where AI models are used:

1. Natural Language Processing (NLP)

• Example models: GPT-4, BERT, T5

• Uses: Chatbots, text summarization, language translation, sentiment analysis, code generation

• Real-world example: AI-powered customer service chatbots and automated content writing tools

2. Computer Vision

• Example models: YOLO, ResNet, Vision Transformers (ViTs)

• Uses: Object detection, facial recognition, medical image analysis

• Real-world example: Security cameras using AI to detect suspicious activity

3. Speech Recognition & Audio Processing

• Example models: Whisper, DeepSpeech, Wav2Vec

• Uses: Voice assistants, transcription services, real-time translation

• Real-world example: AI-powered transcription services for podcasts and meetings

4. Code Generation & Debugging

• Example models: Codex (used in GitHub Copilot), Code Llama

• Uses: AI-assisted programming, bug detection, automated code generation

• Real-world example: Developers using AI to autocomplete and debug code in real-time

5. Image Generation

• Example models: Stable Diffusion, DALL·E, Midjourney

• Uses: AI-generated art, video game design, product concept visualization

• Real-world example: Graphic designers using AI to create unique visuals for marketing campaigns

Given the vast number of specialized AI tasks, a single repository like Hugging Face becomes invaluable for AI practitioners who want to explore and experiment with different models. But with so many contributors and an open-source ecosystem, security vulnerabilities can arise.

How Does Hugging Face Work?

Hugging Face operates as a collaborative model-sharing platform, allowing developers to:

1. Upload AI Models – Users can upload pre-trained models that others can download and use. These models are stored in repositories similar to GitHub.

2. Host and Deploy Models – Hugging Face provides APIs and cloud-based inference endpoints for developers to integrate AI models into their applications.

3. Access Large AI Datasets – The platform includes a large dataset library for training and benchmarking AI models.

4. Use Model Cards & Documentation – Each model repository comes with detailed descriptions, use cases, and implementation examples.

The platform also integrates automated scanning tools to check uploaded models for security vulnerabilities. However, recent events have shown that these security measures are not foolproof.

The Discovery of Malicious AI Models

Image from reversinglabs showing a decompiled pickle file with malicious python code

Recently, cybersecurity researchers discovered that malicious AI models were uploaded to Hugging Face, containing embedded malware designed to execute unauthorized code on users' systems.

According to reports from ReversingLabs and The Hacker News, attackers exploited a weakness in Hugging Face’s security system to distribute malware disguised as machine learning models.

How Did the Attack Work?

1. Malicious Model Upload – Attackers uploaded AI models that contained embedded malware.

2. Pickle File Exploit – These models used Pickle, a Python serialization format that can store arbitrary objects, including executable code.

3. Bypassing Security Scans – Hugging Face uses a tool called Picklescan to detect potentially harmful code inside models. However, the attackers found a way to evade detection.

4. Executing Malware on User Machines – When users downloaded and loaded the infected model, it silently executed malicious code, allowing attackers to gain remote access to their system.

What Makes This Attack Dangerous?

• AI developers trust Hugging Face – Many researchers and organizations blindly trust models on Hugging Face, assuming they are safe.

• Supply Chain Risks – If compromised models are used in critical applications (e.g., healthcare AI, finance, security systems), they could introduce backdoors or vulnerabilities.

• Hard to Detect – The attack method exploits the AI toolchain itself, making it harder to identify compared to traditional malware.

How Did the Malware Evade Detection?

Hugging Face uses Picklescan to analyze uploaded Pickle files for potential security threats. However, the attackers used a compressed file format (7z) instead of the standard ZIP format, which bypassed Picklescan’s detection mechanism.

Key Technical Details of the Attack:

1. Pickle File Manipulation

   • Pickle allows Python objects to be saved and reloaded, but it also allows arbitrary code execution.

   • Attackers embedded a reverse shell payload inside a Pickle object.

2. Use of 7z Compression

   • Hugging Face scans ZIP-compressed files for security risks.

   • Attackers uploaded 7z-compressed models instead, evading Picklescan’s detection.

3. Delayed Execution

   • The malware does not execute immediately. Instead, it waits for deserialization, which occurs when a developer loads the model in their Python environment.

   • This method avoids detection until it is too late.

Final Thoughts

This incident highlights the growing risk of supply chain attacks in AI and machine learning. With AI becoming a core part of modern technology, attackers are finding new ways to exploit the ecosystem.

AI platforms must take proactive steps to improve security measures, and developers must stay vigilant when using third-party models. As AI adoption increases, so will the sophistication of cyber threats targeting the industry.

By raising awareness and improving security practices, the AI community can mitigate risks and build a safer, more trustworthy AI ecosystem.