Introduction:

In the evolving landscape of AI, Large Language Models (LLMs) have become a cornerstone in modern applications, enabling chatbots, APIs, and various interactive systems. However, with great power comes great responsibility — and risk. In this part of our GenAI Cybersecurity series, we will explore how LLMs expand traditional attack surfaces, the security implications for both LLM consumers and providers, and real-world case studies that expose these vulnerabilities.

Let’s start with some basics first

What is an Attack Surface in Cybersecurity?

In cybersecurity, the term attack surface refers to the total number of points where an unauthorized user could potentially access or extract data from a system. Think of it as the sum of all exposed interaction points, such as APIs, open ports, and input fields like login forms or chat interfaces.

Every time we expose a system to external access — whether it’s through a network service, API, or web interface — we are effectively adding a potential entry point that attackers can exploit. This set of exposed elements forms the baseline attack surface.

How Attack Surfaces Expand in Real-World Scenarios

In real-world environments, systems rarely operate in isolation. Organizations integrate third-party tools, mobile apps, and external APIs, which naturally increases the number of interaction points. This is known as the expanded attack surface.

Examples of Expanded Attack Surfaces:
• Exposing CRM data through APIs
• Integrating chatbots that connect to backend systems
• Allowing mobile apps to connect to enterprise servers
• Connecting cloud-based tools to internal databases

While integration provides convenience, it also increases complexity, making it harder to defend the entire attack surface.

Publicly Exposed Attack Surfaces: The Multiplier Effect

The situation worsens when these expanded attack surfaces are publicly exposed. Organizations often expose APIs, chatbots, and services to external networks to allow user interaction, partner integration, or customer support.

However, this openness introduces significant risks. A single misconfigured API or exposed port can become a gateway for attacks.

For instance, attackers can:
• Scan exposed endpoints using automated tools like Shodan.
• Probe for vulnerabilities in API parameters.
• Exploit misconfigurations to gain unauthorized access to backend systems.

Why Large Language Models Have Larger Attack Surfaces

LLMs have a much larger attack surface compared to traditional systems due to the following factors:

• Public Interfaces: LLMs are often accessible through chatbots, APIs, and web apps.
• Unstructured Inputs: They accept user prompts in natural language, allowing attackers to experiment with malicious prompts.
• External Integrations: They are often integrated with APIs, databases, and plugins, increasing the complexity of the attack surface.
• Dynamic Behavior: LLMs evolve based on training data, making their behavior unpredictable and harder to secure.

Two Perspectives on LLM Attack Surfaces:

To effectively understand the security risks, we need to analyze the attack surface from two perspectives:

Scenario 1: LLM Security from the Organization’s Perspective: Renting LLM Model Access through LLM Providers (e.g., OpenAI, Anthropic, Mistral)

Organizations that adopt LLMs for customer interactions, data analysis, or operational support face new security challenges.

Key risks:

• Exposing internal data to external AI models through APIs
• Over-permissive access to databases
• Prompt injections that can manipulate the model’s behavior

These integrations create new endpoints, each of which can be exploited through unauthorized access, data leaks, or prompt injections.

What happens when an org integrates an external LLM into their own ecosystem for customer service.

Real-World Example – Chevrolet Chatbot Prompt Injection:
In 2023, a Chevrolet dealership deployed a chatbot powered by ChatGPT. It was manipulated by a user named Chris Bak, who tricked the chatbot into making a “legally binding” offer for a vehicle at a fraction of its price through prompt injection.

Scenario 2: LLM Security from the Model Provider’s Perspective (e.g., OpenAI, Anthropic, Mistral)

LLM providers host the infrastructure where the models are trained, deployed, and accessed. These are the companies building and hosting LLMs, such as OpenAI and Anthropic. Their focus is on securing infrastructure, APIs, and training pipelines.

Key Risks:

• Unsecured training data pipelines can lead to training data poisoning.
• Inference data storage can expose sensitive data if not properly secured.
• API endpoints can be accessed by unauthorized users, leading to data leakage.

Case Studies in LLM Security – Provider Perspective

1. Microsoft Tay Training Data Poisoning:
In 2016, Microsoft released Tay, a chatbot that learned from user input. Within hours, Tay was manipulated into generating offensive content due to lack of content moderation and input validation.

• Microsoft Tay Chatbot (2016): Attackers exploited Tay on Twitter, feeding it toxic content to manipulate its learning process.
• No Moderation or Filtering: Tay learned directly from public tweets in real-time — without safeguards or oversight.
• AI Started Generating Hate Speech: Within 24 hours, Tay began tweeting racist, misogynistic, and offensive statements.
• Classic Case of Training Data Poisoning: Malicious user inputs became part of the model’s knowledge base, corrupting its behavior.
• How it Could Have Been Prevented: Rate limiting, content filtering, anomaly detection, and human-in-the-loop learning could’ve protected Tay.
• Key Lesson for LLMs: Never let AI learn blindly from user input — always apply guardrails, filters, and human review in production.

2. DeepSeek Log Exposure by Wiz:

In 2024, Wiz security researchers discovered over a million log entries from DeepSeek LLM exposed through an unsecured ClickHouse database. These logs contained sensitive API keys, user prompts, and internal instructions.

• Over 1 million logs exposed publicly by DeepSeek through a misconfigured ClickHouse database.
• No authentication required – anyone could access chat logs, API keys, user prompts, and system instructions.
• Prompt logs included sensitive and private conversations, potentially containing user secrets or business data.
• Log data could reveal internal workings of LLM inference systems – increasing the risk of targeted attacks.
• Attackers could run SQL queries – risking privilege escalation and internal system compromise.
• Discovery took only minutes, highlighting how easy it is to find such exposures with basic scanning tools.
• Wiz reported it responsibly, and DeepSeek secured the leak quickly, but the incident raised serious concerns.
• Lesson: AI systems must treat logs as sensitive data, with access controls, redaction, and monitoring in place.

3. Olama API Exposure:

Olama API provided LLM access to organizations without implementing sufficient access controls. This resulted in 4,000+ servers being publicly exposed on Shodan, allowing anyone to interact with the deployed models.

• Misconfigured Ollama API exposed LLM model servers directly to the internet.
• No authentication or access control allowed anyone to query the models freely.
• First discovered by Reddit users and later investigated by Laava & UpGuard security researchers.
• As of April 2025, my Shodan query from the lab system shows over 2,000 Ollama servers still publicly exposed.
• DeepSeek LLM instances were traced using these exposed APIs.
• Risks include:
• Unauthorized model access
• Potential data leakage via prompts/responses
• Abuse for malicious activities (e.g., jailbreaking the model)
• Highlights the need for securing LLM endpoints like any production API.

Note: Demo of this API Exposure Attack is included in my Udemy GenAI cybersecurity course.

Comparing Responsibilities – User Organizations vs. LLM Providers

Organizations and LLM providers share the responsibility of securing LLM systems. However, their focus areas differ:
• User Organizations: Secure integrations, protect internal data, monitor API traffic, filter inputs.
• LLM Providers: Secure training data pipelines, implement logging controls, restrict model access.

Conclusion

Understanding the attack surfaces of LLM systems requires a dual perspective – one from the organizations using LLMs and another from the providers hosting them. As AI models become more prevalent, securing each layer – from data inputs to integration APIs – will be essential in mitigating emerging threats.

In the next part 4 of this series, we will focus on a specific real-world scenario involving OLAMA server exposures. We will examine why so many OLAMA servers are accessible over the public Internet without authentication and how to implement basic security controls to mitigate this exposure. Stay tuned.

Continue to Part 4: Securing OLAMA Servers — Why They’re Exposed and How to Protect Them