Despite growing adoption, many AI applications are built on false assumptions about model safety, prompt injection, and system behavior. Below this post unpacks the top myths we encounter in the field andwhy real-world threat pressure demands a different approach.

“𝗧𝗵𝗲 𝗺𝗼𝗱𝗲𝗹 𝗶𝘀 𝘀𝗲𝗰𝘂𝗿𝗲, 𝘀𝗼 𝘁𝗵𝗲 𝗮𝗽𝗽 𝗶𝘀 𝘀𝗲𝗰𝘂𝗿𝗲.”

Not even close.

Even a perfectly fine-tuned LLM can be misused in insecure workflows; prompt injection, tool overreach, vector poisoning, and downstream abuse don’t care how safe your base model is. In some cases the larger the model the more easily it can be coaxed into performing undesired behaviour.

“𝗣𝗿𝗼𝗺𝗽𝘁 𝗶𝗻𝗷𝗲𝗰𝘁𝗶𝗼𝗻 𝗶𝘀 𝘀𝗼𝗹𝘃𝗲𝗱.”

It’s not.

Regex filters and system prompts aren’t silver bullets.

Attackers chain context, leverage encodings, embed triggers, poison memory and bypass naive controls in ways many teams haven’t even threat modelled yet. A recent paper from only a few weeks ago found multiple bypass techniques which worked across all tested guardrails. (“Bypassing Prompt Injection and Jailbreak Detection in LLM Guardrails” - https://lnkd.in/g-prgNCM) - One of the most successful uses using emoji variation selectors (aka emoji smuggling) 😲.

“𝗜𝘁’𝘀 𝗷𝘂𝘀𝘁 𝗮𝗻𝗼𝘁𝗵𝗲𝗿 𝗺𝗶𝗰𝗿𝗼𝘀𝗲𝗿𝘃𝗶𝗰𝗲.”

If only.

Traditional authN/Z patterns and input/output validation break down when your app includes a non-deterministic reasoning engine that can interpret context, rephrase inputs, and initiate tool use. AI apps just don’t behave like REST APIs under pressure and can often surprise.

GenAI introduces a new category of dynamic non-deterministic cyber risk, requiring full-stack, continuous, AI-specific security testing.

At Appsurent Cyber Security, we're working with teams to address these myths to help build resilient applications rooted in real-world threat pressure, not hopeful or incomplete assumptions.

Has your organization started integrating adversarial thinking into AI application deployment yet?

‍

Additional reading: https://www.appsurent.com/post/from-sql-injection-to-prompt-injection-why-security-history-repeats-itself