There is no honest single-number answer to whether GPT-4.1 vs DeepSeek V3 Hallucination Comparison.
On Vectara’s current HHEM-2.3 benchmark for source-grounded summarization, GPT-4.1 has the lower measured hallucination rate: 5.6% versus 6.1% for the original DeepSeek-V3. That is a small advantage for GPT-4.1 under the same evaluation framework.
But a peer-reviewed 2026 medical study produced a very different result. DeepSeek-V3 generated zero hallucinated clinical-guideline citations in the evaluated cases, while GPT-4.1 generated hallucinated guidelines in roughly 7% to 8.5% of outputs. The catch is that DeepSeek-V3 rarely supplied the correct guidelines either, with guideline accuracy of only 2.92% in one disease set and 1.84% in another. GPT-4.1 correctly surfaced relevant guidelines much more often.
That creates the most useful conclusion from this comparison:
GPT-4.1 appears slightly more factually consistent in grounded summarization, while DeepSeek-V3 can look safer on some citation-hallucination tests because it is more likely to omit information rather than invent it.
A low hallucination score does not automatically mean a model gives the most complete or useful answer.
GPT-4.1 vs DeepSeek V3 Hallucination Comparison Results
| Evaluation | GPT-4.1 | DeepSeek-V3 | What It Actually Measures |
|---|---|---|---|
| Vectara HHEM-2.3 hallucination rate | 5.6% | 6.1% | Unsupported statements in document summaries |
| Vectara factual consistency | 94.4% | 93.9% | Summary consistency with supplied source |
| Vectara answer rate | 99.9% | 97.5% | How often the model produced an answer |
| Clinical guideline hallucination, hypercholesterolaemia | 7.31% | 0% | Fabricated guideline citations |
| Correct guideline prevalence, hypercholesterolaemia | 46.34% | 2.92% | Valid guidelines included |
| Clinical guideline hallucination, type-2 diabetes | 8.48% | 0% | Fabricated guideline citations |
| Correct guideline prevalence, type-2 diabetes | 69.21% | 1.84% | Valid guidelines included |
These evaluations measure different behaviors. They should not be averaged together or used to produce a universal “hallucination score.”
Why Hallucination Comparisons Are Easy to Misread
A hallucination is generally an output that presents unsupported or false information as though it were factual.
OpenAI describes hallucinations as plausible but false statements and notes that models can produce incorrect definitions, dates, citations, references, and answers even when their wording sounds confident. Its research also argues that conventional accuracy benchmarks can encourage models to guess instead of acknowledging uncertainty. OpenAI’s research on why language models hallucinate
DeepSeek acknowledges the same problem in its official model-disclosure material. The company states that its models may generate incorrect, omitted, or non-factual content and that it cannot guarantee hallucination-free outputs. DeepSeek lists training-data quality, alignment, and retrieval-augmented generation among the approaches it uses to reduce the problem. DeepSeek’s model limitations and training disclosure
But hallucination is only one failure mode.
A model can also:
- Refuse to answer.
- Omit an important fact.
- Give an incomplete answer.
- Cite no source at all.
- Use a correct source incorrectly.
- Provide a factually accurate statement that does not answer the question.
- Give the right conclusion for the wrong reason.
This is why comparing two models only by “how often did they fabricate something?” can reward an overly cautious model.
Independent Summarization Testing Gives GPT-4.1 a Small Edge
Vectara’s hallucination leaderboard is one of the cleaner direct comparisons available because both models are evaluated using the same hallucination-detection framework.
Vectara describes HHEM as an evaluation of how frequently a model introduces unsupported information when summarizing a supplied document. Its current dataset contains more than 7,700 articles across subjects including news, medicine, law, business, technology, science, sports, and education. Vectara Hallucination Leaderboard methodology and results
As of the leaderboard’s May 11, 2026 update:
GPT-4.1
- Hallucination rate: 5.6%
- Factual consistency: 94.4%
- Answer rate: 99.9%
- Average summary length: 91.7 words
DeepSeek-V3
- Hallucination rate: 6.1%
- Factual consistency: 93.9%
- Answer rate: 97.5%
- Average summary length: 81.7 words
The gap is only 0.5 percentage points, so it would be an overreach to describe GPT-4.1 as dramatically more reliable.
The appropriate conclusion is narrower:
For source-grounded document summarization under the current HHEM-2.3 methodology, GPT-4.1 performed slightly better than DeepSeek-V3.
That finding should not automatically be extended to open-ended questions, medicine, coding, legal research, web search, or citation generation.
Why You May See a 3.9% DeepSeek-V3 Hallucination Rate Elsewhere
Older articles about DeepSeek-V3 often quote a 3.9% hallucination rate from Vectara.
That number came from an earlier HHEM evaluation used when Vectara compared DeepSeek-V3 with DeepSeek-R1 in January 2025. At the time, Vectara reported 3.9% for DeepSeek-V3 and 14.3% for DeepSeek-R1. Vectara’s original DeepSeek-V3 and R1 comparison
The current leaderboard uses HHEM-2.3 and a newer dataset, where DeepSeek-V3 is listed at 6.1%.
Do not combine the old 3.9% DeepSeek result with the current 5.6% GPT-4.1 result and call it a fair comparison. They come from different leaderboard generations.
The correct same-benchmark comparison is:
GPT-4.1: 5.6%
DeepSeek-V3: 6.1%
The Clinical Study Tells a Very Different Story
The most interesting direct study of these two exact models was published in BMJ Health & Care Informatics on April 24, 2026.
Researchers generated 13,824 outputs from GPT-4.1 and DeepSeek-V3 using simulated clinical cases involving hypercholesterolaemia and type-2 diabetes. After excluding responses that did not meet the required output format, they analyzed 12,197 outputs. Read the peer-reviewed study via PubMed
The researchers checked whether outputs included real clinical guidelines, omitted relevant guidelines, or cited guidelines that could not be verified.
Hypercholesterolaemia cases
GPT-4.1 included a valid guideline in 46.34% of outputs.
DeepSeek-V3 did so in only 2.92%.
But GPT-4.1 hallucinated clinical guidelines in 7.31% of outputs, while DeepSeek-V3’s hallucinated-guideline rate was 0%.
Type-2 diabetes cases
GPT-4.1 included valid guidelines in 69.21% of outputs.
DeepSeek-V3 included them in only 1.84%.
Again, GPT-4.1 hallucinated some guidelines, at 8.48%, while the DeepSeek-V3 hallucinated-guideline rate remained 0%.
At first glance, that makes DeepSeek look dramatically safer.
It is not that simple.
Zero Hallucinations Can Hide an Omission Problem
Imagine asking two research assistants to identify the relevant government rule for a question.
Assistant A finds the right rule most of the time but occasionally invents one.
Assistant B almost never names a rule at all.
If you measure only fabricated citations, Assistant B looks perfect.
But if your task requires finding the relevant rule, Assistant B may still be much less useful.
That is close to what happened in the clinical study.
DeepSeek-V3’s 0% hallucinated-guideline result was accompanied by extremely low correct-guideline prevalence.
The authors explicitly frame the study around three risks:
- Hallucination
- Omission
- Inconsistency
They also warn that LLM outputs are stochastic, meaning the same underlying task can produce materially different behavior across repeated generations.
This is the key lesson for anyone comparing AI reliability:
A model that says less can achieve fewer hallucinations without necessarily providing more accurate or useful information.
The Clinical Study Does Not Prove GPT-4.1 Is Better at Medicine
The BMJ study deserves careful interpretation.
The researchers state that they were evaluating whether clinical guidelines were correctly identified and cited. They were not measuring overall diagnostic reasoning quality, medical knowledge accuracy, treatment quality, or whether every clinical recommendation was correct.
So the correct conclusion is not:
GPT-4.1 is medically more accurate than DeepSeek-V3.
Nor is it:
DeepSeek-V3 is safer for healthcare because it hallucinated zero guidelines.
The evidence supports something more specific:
GPT-4.1 was substantially more likely to surface real guidelines in this experiment, but it also fabricated some guideline citations. DeepSeek-V3 avoided fabricated guideline citations but frequently failed to provide the expected valid guidelines.
Neither behavior is suitable for unverified clinical decision-making.
OpenAI’s Own Hallucination Tests Show GPT-4.1 Can Be Highly Grounded
OpenAI later published hallucination results for GPT-4.1 while introducing GPT-5.
Without tools, OpenAI reported the following GPT-4.1 hallucination rates:
- LongFact Concepts: 0.7%
- LongFact Objects: 1.1%
- FActScore: 6.7%
OpenAI’s published hallucination benchmark table
These numbers are useful evidence about GPT-4.1, but they are not a direct GPT-4.1 vs DeepSeek-V3 comparison because the same table does not provide DeepSeek-V3 results under identical settings.
They should therefore not be used to claim that GPT-4.1 is several times more factual than DeepSeek.
This distinction between first-party and directly comparable independent evidence matters.
A Better Way to Compare Model Reliability
AI Journal Now’s recommended framework is to judge factual reliability across three separate dimensions, rather than reporting only hallucination rate.
1. Hallucination rate
How often does the model introduce information that is unsupported or demonstrably false?
For the current Vectara summarization test, GPT-4.1 has a slight advantage.
2. Omission rate
How often does the model fail to provide information the task actually requires?
The 2026 clinical study shows why this matters. DeepSeek-V3 produced no hallucinated guideline citations but included surprisingly few valid guidelines.
3. Answer rate and completeness
Does the model answer the question at all, and does it provide enough information to be useful?
Vectara’s current leaderboard lists a 99.9% answer rate for GPT-4.1 and 97.5% for DeepSeek-V3 on its summarization dataset.
A model is not reliable merely because it avoids false statements. It should also provide the correct information when the task calls for it.
Quick Reliability Decision Matrix
| Use Case | Evidence-Based Preference | Why |
|---|---|---|
| Source-grounded document summarization | GPT-4.1, slight edge | 5.6% vs 6.1% HHEM-2.3 hallucination |
| Avoiding fabricated clinical-guideline citations | DeepSeek-V3 in one study | 0% hallucinated guidelines |
| Correctly surfacing clinical guidelines | GPT-4.1 in the same study | Far higher valid-guideline prevalence |
| High-stakes medical, legal, or financial facts | Neither without verification | Both can fail differently |
| Long-context source-grounded workflows | GPT-4.1 has architectural advantages | 1M-token context |
| Open-weight self-hosting | DeepSeek-V3 | Downloadable weights |
| Current new API deployment | Compare newer models instead | DeepSeek-V3 has been superseded |
The clinical rows apply only to the study’s specific healthcare tasks. They should not be generalized to all citation or factuality workloads.
GPT-4.1 and DeepSeek-V3 Are Also Different Architecturally
The models were built under very different deployment models.
OpenAI released GPT-4.1 on April 14, 2025 as a proprietary API model focused on instruction following, coding, and long-context work. Its current API documentation lists a 1,047,576-token context window, up to 32,768 output tokens, image input, function calling, structured outputs, and fine-tuning support. Current GPT-4.1 API documentation
DeepSeek-V3 was released on December 26, 2024 as a 671B-parameter Mixture-of-Experts model with 37B activated parameters per token. The original open model supports a 128K context window. Official DeepSeek-V3 model card and specifications
Those differences do not explain hallucination rates by themselves, but they matter when deciding which model fits a production environment.
Pricing and Availability in August 2026
This comparison has become partly historical because the two developers have moved at different speeds.
GPT-4.1
GPT-4.1 was removed from ChatGPT on February 13, 2026, but OpenAI continues to make the model available through its API. OpenAI’s GPT-4.1 retirement notice for ChatGPT
Current API pricing remains:
- Input: $2.00 per million tokens
- Cached input: $0.50 per million tokens
- Output: $8.00 per million tokens
OpenAI now recommends starting with newer GPT-5-family models for more complex tasks.
DeepSeek-V3
The original DeepSeek-V3 is no longer the current DeepSeek API flagship.
DeepSeek released V3.1 in August 2025, V3.2 in December 2025, and its current transparency center lists DeepSeek-V4, released in April 2026, as the newer generation. DeepSeek’s current model transparency center
DeepSeek’s current API pricing page now lists V4 models rather than the original V3. Current DeepSeek API models and pricing
At its original launch, DeepSeek-V3 was priced at $0.27 per million uncached input tokens, $0.07 for cached input, and $1.10 for output after its promotional period. Those are historical figures and should not be presented as current V3 API pricing.
For a new production system in August 2026, comparing GPT-4.1 with the current DeepSeek V4 generation would usually be more useful than selecting the original V3 based only on older benchmark results.
AI Journal Now’s guide to AI model aggregators and multi-model platforms explains another reason exact model versions matter: aggregators can expose different model snapshots, providers, context limits, and pricing from what is available directly through the developer.
How to Reduce Hallucinations With Either Model
Choosing the lower-hallucination model is only one layer of reliability.
Production systems should also control how the model gets information and what happens when confidence is low.
Ground answers in supplied sources
If the task involves company documents, policies, research, contracts, or current facts, provide the relevant source material or use retrieval rather than expecting the model to recall everything from pretraining.
Require source-backed answers
Ask the system to distinguish statements supported by supplied evidence from statements based on general model knowledge.
Citations still need verification. A model can fabricate a citation or misrepresent a real source.
Allow the model to abstain
OpenAI’s hallucination research argues that evaluation systems often reward guessing rather than uncertainty. A production prompt should make “I don’t have enough evidence” an acceptable answer when evidence is missing.
Verify consequential claims
Medical, legal, financial, safety, and business-critical outputs should not be accepted solely because the model sounds confident.
AI Journal Now’s coverage of AI search engines and their hallucination limitations makes the same practical point for AI-powered search: sourced-looking answers still need verification when the consequences of an error are high.
Evaluate your own task
The most useful hallucination test is built around the workload the model will actually perform.
If your system summarizes insurance documents, test insurance documents.
It extracts regulations, test regulations.
If it generates research citations, verify citations.
A global leaderboard is useful for screening models. It is not a substitute for domain-specific evaluation.
So, Which Model Hallucinates Less?
Based on the strongest directly comparable evidence currently available:
GPT-4.1 has a slight advantage in source-grounded summarization.
Vectara HHEM-2.3 measures GPT-4.1 at 5.6% hallucination versus 6.1% for DeepSeek-V3.
But the 2026 BMJ study shows why that answer needs qualification.
In its clinical-guideline task, DeepSeek-V3 produced no fabricated guideline citations, while GPT-4.1 hallucinated some. Yet DeepSeek-V3 also failed to provide valid guidelines in the overwhelming majority of those cases. GPT-4.1 gave much more useful guideline coverage, at the cost of occasional fabricated citations.
So the practical verdict is:
Choose GPT-4.1 if your priority is stronger source-grounded completeness and slightly better factual consistency under the current summarization benchmark.
Do not choose DeepSeek-V3 solely because one study reported a 0% hallucinated-citation rate. Its omission behavior in that same study is critical context.
And for any high-stakes workflow, neither model should be trusted without retrieval, source verification, and human review.
The most reliable model is not simply the one that invents the fewest facts.
It is the one that gives the correct information, avoids unsupported claims, knows when evidence is missing, and performs consistently on the task you actually need to automate.
