AI Detection Model Accuracy: WordBinary’s Latest Model Tested on 9,000 AI Texts and 2,000 Human Papers

AI detection model accuracy has become an important concern for universities, students, researchers, publishers, and professional organisations as generative AI systems become more fluent and widely used. A reliable AI detection model must not only identify AI-generated writing with a high recall rate, but also protect genuine human writing from false positives. This study evaluates WordBinary’s latest AI detection model using a dataset of 11,000 text samples, consisting of 9,000 AI-generated texts and 2,000 pre-2015 human academic paper samples.

The AI-generated dataset included outputs from Claude, Cohere, DeepSeek, Gemini, Mistral, OpenAI, and OpenRouter. The test set included both normal AI outputs and humanised AI outputs to assess whether the model could identify AI-generated writing after stylistic rewriting or humanisation. The human-written dataset included pre-2015 academic paper samples from computer science, economics, statistics, physics, and mathematics. These texts were used to evaluate the model’s ability to protect genuine human academic writing from incorrect AI classification.

The results show that WordBinary’s latest AI detection model correctly detected all 9,000 AI-generated samples as AI and correctly classified all 2,000 human academic samples as human. The model achieved 100% AI recall across the tested AI-generated dataset and produced no false positives in the tested human academic paper dataset. The highest AI score recorded among the human academic samples was 1.07%, indicating strong human text protection within the scope of this evaluation.

The findings suggest that WordBinary’s latest AI detection model demonstrated strong performance across AI-generated text detection, humanised AI detection, and human academic text protection in this dataset. The results also show the importance of evaluating AI detection model accuracy through both recall and false-positive behaviour rather than relying on a single AI score alone.

1. Introduction

AI detection model accuracy is now central to academic integrity, research assessment, digital publishing, and professional document review. As generative AI tools become more capable, the boundary between human-written and AI-generated text has become harder to evaluate through surface-level reading alone. AI systems can now produce essays, research summaries, reports, explanations, blog articles, technical notes, and business documents that appear fluent and coherent. At the same time, genuine human academic writing may also appear highly structured, formal, and polished.

This creates a practical challenge for AI detection systems. A model must detect AI-generated text across different platforms and writing styles while avoiding incorrect classification of genuine human work. A tool that only performs well on obvious AI output is limited. A tool that catches AI text but frequently marks human writing as AI is also unsuitable for serious academic or professional use. Strong AI detection model accuracy requires a balance between detection sensitivity and human text protection.

This study evaluates WordBinary’s latest AI detection model against a large mixed dataset of AI-generated and human-written academic samples. The evaluation used 9,000 AI-generated text samples and 2,000 pre-2015 human academic paper samples. The AI-generated samples were produced across multiple AI platform groups, including Claude, Cohere, DeepSeek, Gemini, Mistral, OpenAI, and OpenRouter. The dataset included both normal AI writing and humanised AI writing.

The human-written dataset was selected from pre-2015 academic paper samples across five domains: computer science, economics, statistics, physics, and mathematics. Pre-2015 academic writing provides a useful reference category because these texts predate the widespread availability of modern generative AI systems. The human dataset was therefore used to evaluate whether the model wrongly assigned AI probability to genuine academic writing.

The study focuses on three main areas: AI recall, missed AI classification, and human false-positive behaviour. AI recall measures how many AI-generated samples were correctly detected as AI. Missed AI classification measures how many AI-generated samples were incorrectly treated as human. Human false-positive behaviour measures whether genuine human academic samples were incorrectly classified as AI.

The results show that WordBinary’s latest AI detection model correctly classified every sample in the supplied evaluation dataset. All 9,000 AI-generated texts were detected as AI, and all 2,000 human academic samples were kept as human. These findings support strong AI detection model accuracy within the scope of the tested dataset.

2. Study Objective

The objective of this study was to evaluate the AI detection model accuracy of WordBinary’s latest detection system across a large dataset containing both AI-generated and human-written academic text samples.

The study was designed to assess whether the model could:

1. Detect AI-generated text from multiple AI platform groups.

2. Detect both normal AI and humanised AI writing.

3. Correctly classify pre-2015 human academic paper samples as human.

4. Maintain low AI scores on genuine human academic writing.

5. Demonstrate strong AI recall without increasing false positives on human text.

The evaluation measured performance at both dataset level and platform level. The dataset-level analysis examined total AI detection and total human classification results. The platform-level analysis examined AI recall and average AI scores across Claude, Cohere, DeepSeek, Gemini, Mistral, OpenAI, and OpenRouter. The model-level analysis further examined specific models and modes, including humanised and normal AI outputs.

This approach allows the study to move beyond a single overall percentage and examine how the model performed across different AI generation sources and writing conditions.

3. Dataset Description

The dataset contained 11,000 total text samples. It was divided into two main groups: AI-generated texts and human pre-2015 academic paper samples.

The AI-generated group included 9,000 samples from seven platform categories. These samples represented outputs from major AI systems and model providers. The dataset included both direct AI writing and humanised AI writing, allowing the evaluation to test AI-generated content detection under different output conditions.

The human-written group included 2,000 pre-2015 academic paper samples. The samples covered five academic domains:

The inclusion of multiple academic domains is important because academic writing patterns differ across disciplines. Mathematics and physics often include concise technical language, equations, definitions, and structured explanation. Computer science papers may include method descriptions, algorithmic language, and formal terminology. Economics papers may include theoretical framing, statistical explanation, and policy discussion. Statistics papers often contain methodological precision and repeated technical terms.

A reliable AI detection model must avoid treating these academic conventions as AI signals by default. The human paper dataset therefore provides a useful test of human academic text protection.

4. Methodology

Each sample in the dataset was processed using WordBinary’s latest AI detection model. The model produced an AI classification and AI score for each sample. The classification result was compared against the known label of the sample.

For AI-generated texts, a correct result meant that the sample was detected as AI. For human-written academic paper samples, a correct result meant that the sample was classified as human.

The study used four main evaluation measures.

4.1 AI Detection Recall Rate

AI detection recall rate measures the proportion of AI-generated samples correctly detected as AI.

The formula used is:

AI recall = correctly detected AI samples / total AI-generated samples

In this study, the model detected 9,000 out of 9,000 AI-generated samples.

4.2 Missed as Human

This measure counts the number of AI-generated samples that were incorrectly classified as human. A high number of missed AI samples would indicate weak AI detection performance. In this study, no AI-generated samples were missed.

4.3 Human Correct Classification

This measure counts the number of human-written samples correctly classified as human. It is an important measure because AI detection systems must protect genuine human writing from incorrect AI classification.

In this study, the model correctly classified all 2,000 human academic samples as human.

4.4 AI Score on Human Writing

The AI score assigned to human writing was examined to understand false-positive risk. A sample may technically be classified as human while still receiving a moderately high AI score. For that reason, the study also reviewed the average AI scores across human academic domains and the highest AI score in the human dataset.

The highest AI score recorded in the human academic set was 1.07%, indicating very low AI attribution for genuine human paper samples in this evaluation.

5. Overall Results

The overall results show that WordBinary’s latest AI detection model correctly classified every sample in the supplied dataset.

The AI-generated group produced a 100% AI detection recall rate. All 9,000 AI-generated texts were detected as AI.

The human-written group produced a 100% correct human classification result. All 2,000 pre-2015 human academic paper samples were classified as human.

The combined result shows strong AI detection model accuracy in the tested dataset. The model demonstrated high sensitivity to AI-generated text while maintaining strong protection for genuine human academic writing.

This balance is important. In AI detection, a high AI recall rate alone is not sufficient. A model may detect AI aggressively but wrongly flag human-written text. Similarly, a model may avoid false positives by being too lenient and missing AI-generated samples. In this evaluation, WordBinary’s latest model achieved both high AI recall and strong human text protection.

6. AI Platform-Level Results

The AI-generated dataset included outputs from Claude, Cohere, DeepSeek, Gemini, Mistral, OpenAI, and OpenRouter. Each platform group was evaluated for sample count, detected AI count, missed AI count, recall, and average AI score.

Across all seven platform groups, the model detected every AI-generated sample. No samples were missed as human.

OpenAI outputs recorded the highest average AI score at 99.98%. Cohere followed closely at 99.94%, Gemini at 99.89%, Claude at 99.48%, Mistral at 99.41%, OpenRouter at 99.30%, and DeepSeek at 98.56%.

All platform-level average AI scores were above 98%. This indicates that the model did not merely classify samples as AI at a borderline level. It assigned consistently high AI scores to AI-generated texts across all tested platform groups.

The lowest platform-level average AI score was recorded for DeepSeek at 98.56%, which remains high. This suggests that WordBinary’s latest AI detection model maintained strong detection confidence even on the platform group with the lowest average score in the dataset.

7. Per-Model and Mode-Level Results

A more detailed evaluation was conducted at model and mode level. The dataset included normal AI outputs and humanised AI outputs. This distinction is important because humanised AI writing can reduce visible AI patterns and make detection more difficult.

The per-model results show 100% recall across every listed model and mode. No model group produced missed AI samples.

The lowest average AI score in the AI-generated dataset was 94.98%, recorded for DeepSeek Reasoner humanised samples. Even in this group, all 400 samples were detected as AI. The highest average AI score was 100.00%, observed across several normal AI and humanised AI groups.

The results show that the model performed consistently across both normal AI and humanised AI modes. Normal AI outputs generally produced very high AI scores, often reaching 100.00%. Humanised outputs also produced high AI scores, although some groups showed lower average AI text percentages. This suggests that humanisation reduced certain AI-text percentage indicators in some cases but did not prevent the model from classifying the samples as AI.

The model’s performance on humanised AI samples is particularly relevant because humanised AI content is more challenging than direct AI output. Humanised AI writing may include more varied phrasing, less predictable sentence structure, and fewer obvious AI-style transitions. In this study, WordBinary’s latest model maintained complete recall across the tested humanised AI samples.

8. Human Academic Paper Results

The human-written dataset included 2,000 pre-2015 academic paper samples across five domains: computer science, economics, statistics, physics, and mathematics.

All 2,000 human academic samples were correctly classified as human. No human sample was classified as AI. The highest human AI score was 1.07%.

The average AI scores across the human domains were very low. Mathematics recorded the lowest average AI score at approximately 0.01%. Computer science recorded approximately 0.05%, statistics approximately 0.06%, physics approximately 0.06%, and economics approximately 0.31%.

The economics category produced the highest average AI score among the human domains, but the score remained far below a level that would indicate AI authorship. This suggests that the model did not treat formal academic argumentation, structured analysis, or discipline-specific writing as AI-generated by default.

The human academic paper results are significant because false positives are one of the most serious risks in AI detection. Academic writing often follows standard conventions. It may contain formal language, repeated terminology, cautious claims, structured paragraphs, and discipline-specific phrasing. These features should not be treated as sufficient evidence of AI generation.

In this evaluation, WordBinary’s latest AI detection model showed strong human academic text protection across all five tested domains.

9. Analysis of AI Recall

AI recall is one of the most important measures of AI-generated text detection. It indicates how effectively a model identifies AI-generated writing when the text is known to be AI-generated.

In this study, the AI detection recall rate was:

9,000 detected AI samples / 9,000 AI-generated samples = 100% AI recall

A 100% recall result means that no AI-generated sample in the test dataset was missed as human. This result was consistent across all seven platform groups and across all listed model and mode combinations.

The platform-level recall results were:

The recall result is supported by the average AI scores. All platform-level average AI scores were above 98%, which indicates that classifications were not weak or marginal at platform level. This strengthens the interpretation that the model consistently recognised AI-generated patterns across the tested sources.

AI recall is especially important in academic and professional settings because missed AI samples may lead to incorrect acceptance of AI-generated work as human-authored. The result from this study suggests that WordBinary’s latest model performed strongly in identifying AI-generated writing across the tested AI platform groups.

10. Analysis of Human False-Positive Protection

Human false-positive protection measures whether a detector wrongly labels human-written text as AI-generated. This is a critical part of AI detection model accuracy because false positives can harm genuine writers.

In this study, the false-positive result was:

0 false positives / 2,000 human academic samples = 0% false-positive rate in the tested human set

Every human academic sample was correctly classified as human. The highest AI score among human samples was only 1.07%, which indicates that the model assigned very low AI probability to genuine human academic writing in this dataset.

This result is important because a detector can appear strong if it is tested only on AI-generated text. However, a complete evaluation must also test human writing. A model that identifies AI text but wrongly flags human writing cannot be considered reliable for academic or professional review.

The human academic paper results show that WordBinary’s latest model protected human writing across the five tested domains. This supports the model’s suitability for document review contexts where both AI detection and fair treatment of human authors are important.

11. Normal AI and Humanised AI Comparison

The dataset included both normal AI and humanised AI outputs. This distinction allows the study to examine whether the model’s performance was limited to direct AI-generated text or remained strong after the AI text was modified.

Normal AI outputs generally produced very high average AI scores, often reaching 100.00%. Humanised AI outputs also produced high average AI scores across all tested model groups.

Examples from the humanised AI results include:

All listed humanised AI groups were detected with 100% recall. The result suggests that humanisation did not prevent detection within this test set.

Humanised AI detection accuracy is an important component of AI detection model validation. Humanised text may reduce some obvious surface-level signals, but deeper statistical, structural, and linguistic patterns may still remain. The WordBinary model’s performance on humanised AI samples suggests that its detection approach captured signals beyond simple surface phrasing in this dataset.

12. Interpretation of Average AI Scores

The average AI score provides additional context beyond binary classification. A model may correctly classify a sample as AI but assign a lower confidence score. Conversely, a model may classify human writing as human but still assign a moderately high AI score. For this reason, score distribution matters.

In the AI-generated dataset, average platform-level AI scores ranged from 98.56% to 99.98%. These scores indicate strong AI attribution across all platform groups.

In the human academic dataset, average AI scores ranged from approximately 0.01% to 0.31% by domain. The highest individual human AI score was 1.07%. These values indicate very low AI attribution for genuine human academic writing.

This wide separation between AI-generated scores and human academic scores is one of the strongest findings in the study. The AI-generated samples received very high AI scores, while the human academic samples received very low AI scores. This separation suggests strong discriminative performance in the tested dataset.

The score pattern also supports the model’s practical usability. In document review, the distance between AI and human score distributions can help reduce ambiguity. When AI-generated texts cluster near high scores and human texts cluster near very low scores, users receive clearer signals. In this study, the score separation was substantial.

13. Discussion

The results of this study indicate strong AI detection model accuracy for WordBinary’s latest model within the tested dataset. The model detected all 9,000 AI-generated samples and correctly classified all 2,000 human academic samples.

The findings are notable for three reasons.

First, the AI-generated dataset was not limited to one AI provider. It included outputs from Claude, Cohere, DeepSeek, Gemini, Mistral, OpenAI, and OpenRouter. The model maintained 100% recall across all platform groups.

Second, the dataset included humanised AI outputs. Humanised AI writing can be more difficult to detect because it may contain more natural phrasing and fewer obvious AI-like patterns. The model detected every humanised AI sample in the supplied dataset.

Third, the human-written dataset showed very low AI scores. All 2,000 pre-2015 human academic samples were correctly classified as human, and the highest AI score among human samples was only 1.07%.

Together, these results show that the model performed strongly on both sides of the AI detection problem. It detected AI-generated writing while protecting human academic writing from false positives.

The study also reinforces the importance of evaluating AI detection tools using multiple measures. Accuracy alone can be misleading if the dataset is imbalanced or if only AI-generated samples are tested. A stronger evaluation should include AI recall, missed AI results, human false positives, average AI scores, and domain-level performance. This study included those measures and found consistent performance across the supplied dataset.

14. Practical Implications

The findings have practical implications for students, universities, researchers, and professional users.

For students, the results suggest that WordBinary’s latest AI detection model can provide a strong review signal for AI-generated writing. Students using AI detection before submission can use the result to identify whether their document contains AI-like patterns that require review or revision.

For universities, the results support the use of AI detection as part of an academic integrity workflow. The human academic paper results are particularly relevant because they show that the model did not falsely classify the tested pre-2015 human academic samples as AI.

For researchers, the study provides evidence of model performance across both AI-generated and human-written academic samples. The inclusion of multiple AI platform groups and multiple academic domains strengthens the usefulness of the evaluation.

For businesses and professional organisations, the results show potential value for content review, authorship verification, editorial quality control, and policy compliance. The combination of AI recall and human text protection is especially relevant where incorrect classification may affect trust or decision-making.

The findings also support a broader document review approach. AI detection is one part of text evaluation. Plagiarism checking, grammar checking, citation review, source analysis, and human judgement remain separate but complementary parts of responsible document assessment.

15. Limitations

This study presents strong results within the supplied dataset, but the findings should be interpreted within the scope of the evaluation.

The dataset tested AI-generated samples from selected model groups and human academic samples from selected domains. Performance may differ with other languages, very short text, heavily edited mixed-authorship documents, translated writing, informal writing, creative writing, OCR-extracted text, or documents containing substantial quoted material.

The study focused on AI-generated samples and human-written samples as separate categories. Real-world documents may contain a mixture of human and AI writing. A document may include human-written sections, AI-generated paragraphs, grammar-corrected passages, paraphrased content, and quoted source material in the same submission. Mixed-authorship analysis remains an important area for further evaluation.

The human dataset focused on pre-2015 academic paper samples from computer science, economics, statistics, physics, and mathematics. These domains provide useful academic coverage, but future evaluation across additional fields such as law, medicine, education, management, humanities, and social sciences would provide broader evidence.

The results should therefore be understood as a strong evaluation outcome for the tested dataset rather than a claim of universal accuracy across every possible writing condition.

16. Future Research Directions

Future research can extend this evaluation in several ways.

First, mixed human-AI documents should be tested. A useful future study could evaluate documents where 10%, 25%, 50%, and 75% of the content is AI-generated. This would help measure how the model responds to partial AI use rather than fully AI-generated or fully human-written text.

Second, grammar-corrected and AI-polished human writing should be evaluated separately. Many writers use grammar tools to improve clarity without using AI to generate original content. Testing this category would help distinguish between language correction and AI authorship.

Third, more academic disciplines should be included. Future datasets could include humanities, law, medicine, education, management, psychology, sociology, and business studies. This would allow wider domain-level validation.

Fourth, multilingual and regional English writing should be tested. Indian academic English, non-native English, translated English, and multilingual writing patterns may present distinct detection challenges.

Fifth, future studies can examine sentence-level and document-level score alignment. A document-level AI score is more useful when users can understand which sections contributed to the result and why those sections may require review.

Sixth, additional humanisation and paraphrasing conditions should be tested. Humanised AI can be produced through manual rewriting, paraphrasing tools, AI rewriting prompts, or dedicated humaniser tools. Testing multiple humanisation methods would provide deeper insight into model robustness.

17. Conclusion

This study evaluated WordBinary’s latest AI detection model using 11,000 text samples, including 9,000 AI-generated texts and 2,000 pre-2015 human academic paper samples. The aim was to assess AI detection model accuracy through both AI recall and human false-positive protection.

The model detected all 9,000 AI-generated samples as AI, producing a 100% AI detection recall rate in the supplied dataset. It also correctly classified all 2,000 human academic samples as human, producing no false positives in the tested human set. The highest AI score recorded among human academic samples was 1.07%, while platform-level average AI scores for AI-generated text remained above 98% across all tested AI platform groups.

The AI-generated dataset included outputs from Claude, Cohere, DeepSeek, Gemini, Mistral, OpenAI, and OpenRouter. The model maintained complete recall across all platform groups and across both normal AI and humanised AI modes. The human academic dataset covered computer science, economics, statistics, physics, and mathematics, with all domains correctly classified as human.

The findings show strong AI detection model accuracy within the scope of this evaluation. WordBinary’s latest model demonstrated high AI-generated text detection performance, strong humanised AI detection performance, and strong protection of genuine human academic writing.

The study also highlights the importance of evaluating AI detection systems through multiple measures. AI recall, missed AI samples, human false-positive rate, domain-level performance, and score separation all provide important context. A reliable AI detection model should not only detect AI-generated writing; it should also avoid incorrectly flagging human authors.

Within the tested dataset, WordBinary’s latest AI detection model achieved this balance.