AI Safety & Security: what is known

Last reviewed Oct 3, 2026. Safety and security explainers, not advice for any individual situation.

The short version

• Chatbots state false things fluently. Even the makers say hallucinations are a fundamental challenge for all large language models.¹
• Fake voices, images and video are used in fraud now. The FBI has warned the public about it, and the International AI Safety Report says harms are well documented while data on how common they are is limited.^2,3
• AI tools that read untrusted content can be tricked. The UK’s NCSC says prompt injection may never be fully fixed the way older web flaws were, so risk has to be limited by design.⁴
• What you type can be stored. The NCSC notes that a chatbot company can see the questions you ask it, store them, and will almost certainly use them to develop its service at some point.⁵ Settings differ by app: see Chat privacy.

Hallucinations

OpenAI defines hallucinations as “plausible but false statements generated by language models”. Its explanation: models first learn by predicting the next word in huge amounts of text, with no true/false labels attached, so arbitrary low-frequency facts (its example is a person’s birthday) cannot be predicted from patterns alone. It adds that most tests score only accuracy, which rewards guessing over saying “I don’t know”.¹

OpenAI also gives an example from its own tests, below. It is a company result on one quiz, not a general error rate.¹

The International AI Safety Report 2026 says AI systems can generate non-existent citations, biographies and facts, that human oversight helps but invites over-reliance when answers are fluent and confident, and that human verification of outputs remains necessary in high-stakes settings such as medicine and finance.³

What the science says is known, and not known

The International AI Safety Report 2026, written by over 100 independent experts with an advisory panel nominated by more than 30 countries and organizations, sorts risks into malicious use, malfunctions and systemic risks.³ A reading of its executive summary:

On what companies and governments are doing: 12 companies published or updated Frontier AI Safety Frameworks in 2025, most risk management remains voluntary, and a few jurisdictions are beginning to make some practices legal requirements. The report says technical safeguards are improving but that determined users can still sometimes get harmful outputs by rephrasing or splitting requests.³

Deepfakes and AI scams

The FBI’s December 2024 public alert describes how criminals use generative AI to make fraud more believable and to work at larger scale: text for fake profiles and fraud sites, images for fake IDs and profile photos, short cloned voice clips of a loved one in a made-up emergency, and video for real-time calls posing as executives or officials. It notes that creating or sharing synthetic content is not by itself illegal, but can be used for crimes such as fraud and extortion.²

The FBI’s listed protections, paraphrased:

• Create a secret word or phrase with your family to verify identity.²
• Hang up and call the person or organization back on a number you looked up yourself.²
• Never send money, gift cards, cryptocurrency or other assets to people you have met only online or by phone, and do not share sensitive information with them.²
• Limit public photos and voice recordings where you can, make social accounts private, and limit followers to people you know.²
• Look for flaws in images and video (distorted hands, odd teeth or eyes, wrong shadows, lag, lip-voice mismatch), but the FBI itself says AI content can be hard to spot.²

For the FBI and FTC loss figures and the full list of habits, see AI scams and deepfakes. To report fraud to the FBI, use the IC3 site named in the alert.²

Prompt injection and AI agents

Prompt injection is when instructions hidden in content an AI reads (a web page, email or document) change what the AI does. OWASP ranks it first among its risks for large-language-model applications, and says research shows common fixes such as retrieval and fine-tuning do not fully prevent it.⁶

The NCSC explains why it is hard: inside a language model there is no separation between “data” and “instructions”, only the next token, so prompt injection may never be totally mitigated the way SQL injection can be. Its advice for builders is to reduce likelihood and impact, to use safeguards that do not rely on the model, and to give the AI no more privileges than the least-trusted party whose content it reads.⁴ OWASP likewise recommends least-privilege access and human approval for high-risk actions.⁶

Privacy tips

The NCSC’s guidance, written in 2023 but still on its site, says a language model does not automatically add what you type to its knowledge for other people to query. However, your query is visible to the company providing the service, is stored, and will almost certainly be used for developing the service or model at some point, and the company or its partners may be able to read it. It advises understanding the terms and privacy policy before asking sensitive questions.⁵

• Leave out passwords, ID and account numbers, health, legal and financial records, other people’s details and confidential work files. Use placeholders such as [Name].
• Check the data and training settings in each app. We list them app by app on Chat privacy.
• Business and school accounts can follow different rules from personal ones; see consumer vs business accounts.
• Terms: data retention, personally identifiable information, zero data retention.

Who sets guidance

In the United States, NIST publishes a voluntary AI Risk Management Framework (January 2023) and a Generative AI Profile (NIST AI 600-1, July 26, 2024) intended to help organizations build trustworthiness into AI products.⁷ See the glossary entry on the NIST AI RMF. For laws, see AI rules and jobs and AI by Country.

Keep reading

• Try it: Reality check prompts
• AI scams and deepfakes
• Chat privacy
• How AI works
• Myths vs Facts
• Glossary: AI safety