How Reliable Are AI Detectors For Academic Text?

Many students and researchers use ChatGPT to help write manuscripts, but some grow concerned about being unintentionally flagged as AI-generated. AI-assisted writing can help improve grammar, phrasing, and logic, as well as save significant time in the writing process. But, using these AI tools risks your content is flagged as AI-generated plagiarism. Here’s how AI detectors work, their reliability, and how to use AI ethically and responsibly in order to improve your research productivity without compromising your academic integrity or getting unintentionally flagged.

After publishing a post on using AI to “peer-review” my unpublished manuscript, someone reached out to me asking this:

I think that there is a belief that once you upload something “to AI,” it becomes part of it, and detectors can pick it up later. This is a misunderstanding. In this article, I’ll explain exactly how AI detection works, how reliable it is and to what extent using AI can be considered plagiarism. We will focus on academic text and, in the end, discover the best ways to responsibly use AI for academic writing.

How does AI detection work?

AI detection works in two ways that may be combined for individual detectors: Feature-based and model-based. Feature-based approaches look at specific characteristics of a text, and measure the overall text based on statistical properties. Model-based approaches look at the text more wholistically, and thus tend to be even more accurate. Both are discussed below.

1. Feature-based: These algorithms measure statistical properties (or features) of the text. One commonly measured feature is perplexity. Perplexity is the amount of randomness in the text. A sentence like “I was drinking coffee” is more common than “I was drinking blood” and thus would have a lower perplexity score. If you look at the randomness of sentence structure rather than single words you get to a feature called burstiness. AI models will always lean towards average and consistent structure for ~15 words (in English), while humans will show more variability.
2. Model–based: This is a more complex approach where a machine learning model is presented with AI and human-generated texts and needs to learn to distinguish between these two. The model itself is built very similarly to large language models (i.e. the AI generating the text) themselves. While you need much more data to train these, they can learn to identify features like “burstiness” and perplexity, as well as many others. Model-based approaches pick up on more subtle patterns than feature-based methods.

Individual products for AI detection, like Winston, may combine these methods and invent their own. Check out Prova 2024 for a more thorough review and comparison of the methods.

💡 The key takeaway here is that AI generation works by analysing the structure of the text, not comparing it to existing texts. It fundamentally differs from plagiarism detection, which compares a text to existing sources.

Are AI detectors better than before?

AI detectors are not necessarily better today than they were when they first came out.

As time passes, we’d imagine that AI detectors improve and become even more accurate and reliable. However, AI detectors are trained on AI-generated text and, therefore, always lag behind AI models. If a new model comes out that improves the similarity to human texts, AI detectors will need to be retrained to catch up. When it comes to how good AI detectors really are, it depends on several factors (answered below).

Is AI detection reliable?

2026 Update

AI detectors have improved since the first tools appeared, but newer research still suggests that their results should be treated as indicators rather than proof of AI use. A 2026 study comparing Turnitin, GPTZero, Copyleaks and Pangram found enormous differences between tools. Pangram performed comparatively well, while the other detectors substantially underestimated AI-generated content, especially text produced by newer AI models. Another 2026 study found overall accuracies of only 61% for Turnitin and 69% for Originality.ai, with both struggling particularly with text that combined human and AI writing.

AI detection can work reliably, but it depends on which tool you use.

Some AI detectors are more reliable than others, and when compared systematically, they do work. However, finding if a single text is AI-generated is difficult. Here is a quick example. I asked ChatGPT to summarise my paper and pasted the text into ZeroGPT and WinstonAI, two popular AI detectors. ZeroGPT got it right, and could tell that the text was 0% human, whereas WinstonAI believed it was “likely human written”, with only 55% coming from AI. Interestingly, I did this experiment one year ago, and at that time, the tool results were opposite.

Not only are the AI detectors not accurate, they are also not consistent. This is because the AI models evolve, and only if the tool itself rapidly evolves to meet the changing AI-text generation can it hope to accurately detect the text.

On average, numerous studies support the claim that AI detection is unreliable. Below is an image of the accuracy of various tools from one of the most cited AI detection studies (Weber-Wulff 2023), as of 2023. Accuracy of AI detector tools may vary based on the underlying model and what is being tested. For example, a study focusing on grading English as a second language essays found that these tools were generally reliable (Ibrahim 2023). Keep in mind, tools need to evolve as fast as the AI models do, so just because a tool was reliable in 2023 does not mean it will continue to do so in 2026.

Back in 2023, the authors concluded their work with a stark warning that academics shouldn’t rely on any of these tools:

“Our findings do not confirm the claims presented by the systems. They too often present false positives and false negatives. Moreover, it is too easy to game the systems by using paraphrasing tools or machine translation. Therefore, our conclusion is that the systems we tested should not be used in academic settings.” 

One of the most reliable tools for AI detection is Originality.ai’s AI Checker – an AI detection tool evaluated in ten third-party studies and has shown comparatively strong performance in detecting academic and paraphrased content. However, Originality.ai offers no free trial.

Keep in mind, most discussions about AI detection accuracy is based on the mainstream commercial tools available online. Independent researchers build AI-detection models, which tend to be much more powerful and accurate. For example, in this recent 2024 paper, the researchers attained 98% accuracy on their test set!

AI Detectors vs. Plagiarism Checkers

AI detection and plagiarism detection are fundamentally different tools, testing for independent things. AI detectors analyse the statistical patterns of text to determine whether it is AI-generated or not. Plagiarism detection works by comparing passages from your text with text published elsewhere. Sometimes these models may “bleed” into one another, and you’ll often find that AI detection tools also have plagiarism-check features. Since AI models are trained on published text, they may inadvertently plagiarise if they recreate some input text in their replies.

Using AI, however, does not necessarily lead to plagiarism. For example, one of the most popular tools for plagiarism detection is TurnItIn, which is integrated into Paperpal, an AI writing assistant. Paperpal and TurnItIn are intended to work together. Read everything about these tools in my deep-dive on Paperpal. Turnitin analyses sections of the text and matches it to sources it found online.

When is using AI considered plagiarism?

AI models are trained on others’ work, and their output can sometimes include parts that can be only slightly paraphrased from the original text. Insufficiently paraphrasing someone’s work, even if attributing it to them, is considered plagiarism (see this guide from Harvard). Moreover, AI often cannot attribute the sources correctly. This means that by using AI-generated content, you may unintentionally plagiarise (learn more here).

However, you also do not need to entirely avoid using AI for writing. AI is a powerful and transformative tool that dramatically improves productivity and saves crucial time. The key is to use it responsibly.

Will uploading unpublished work to AI flag it as AI content later?

No. Generally, uploading your content anywhere will not make it more or less probable to be detected as AI. This is because an AI detector is a tool that analyses the structure of your text rather than comparing it with existing texts. There is no online content database where all AI content runs together; quite the opposite, companies will try to hold on to human-generated content, as it becomes a precious commodity on which to train new models. Since the model cannot know if what you upload is human or AI-generated, it’s unlikely to used as human-generated training data.

Do AI detectors steal your writing?

One of the biggest concerns about using AI for academic writing is privacy. Anything you upload can be used in the future for AI training. Social networks have been collecting similar data for quite a while now. Long conversations like those with ChatGPT are a goldmine for content that can be used to train future models. Here is an example from LinkedIn collecting your data for training AI models (Turn it off by navigating to your profile settings and clicking on “Data privacy”, then on “Data for Generative AI Improvement”)

Naturally, if you upload sensitive information, a future ChatGPT version might present it to someone who asks for it. Hopefully, this should not be the case for any manuscript you are aiming to publish, but it is a concern if you upload raw materials like patient information, locations of endangered species, detailed information on specific people, and so on.

The case is similar for AI detectors. These tools also have the ability to save and train on your data, although they are not “stealing” it. If you read the Terms & Conditions of most AI tools, they do reserve the right to use the data you upload.

While your upload likely will not make a difference in the big picture, it is still worth noting that depending on the service and your settings, content you upload to an AI tool may be used to improve its models. So, to be on the safe side, it’s best only to upload what you want to publish anyway.

Can you avoid AI detection?

Turning AI-generated text into text that is indistinguishable from human-generated text is a quickly evolving field. Although there are some valid applications, these tools can naturally be used unethically to bypass AI detectors in contexts where AI-generated text is not desired. It can be important for instructors and educators to understand these methods, since they could be used by students to submit fully AI-generated assignments without detection.

Tools to avoid AI detection attempt to modify properties that the AI detectors analyse. This is called “humanising” the text. Today, people experiment with specific prompts, custom GPTs or by using tools like WriteHuman that target specific AI detectors like Turnitin and GPTZero. Humanising and AI detection is an evolutionary cat-and-mouse game where each tool tries to outperform its competitors.

In my original experiment, transforming the text with WriteHuman or the prompt in the Reddit thread was not enough to fool Winston into believing the text was human-generated. These tools, like AI detectors themselves, are not always reliable.

How to Write with AI (Ethically)

So far in this article we have established that neither AI detection, nor its counterpart (humanising text) are reliable. Furthemore, using AI sometimes runs the risk of unintentional plagiarism. So, what is best way to use AI for writing?

First of all, AI can do much more than just writing. In fact, AI can radically transform each step of the research process, from gathering literature to analysing papers to polishing a final manuscript. If you wish to know more about using AI in the context of academia, I strongly recommend you join my 21-day free course which covers this amongst many other topics.

👉 The newly released (2026) AI Agents Course brings you right into the action, building a full-fledged AI-powered research system with the latest tech. It is not a beginners course, however, and requires some prior understanding of Obsidian workflows. Check out our courses page for more info.

When it comes to using AI responsibly for academic writing, there is a right way. Here are some of the best, ethical applications of AI when it comes to accelerating your writing:

1. Ask AI to provide alternatives for clunky sentences you can’t just get right.
2. Use AI to review your manuscript and find flaws in your writing.
3. AI can teach you to write better by analyzing reviewer comments.
4. Use ChatGPT output as a scaffold for your writing and edit the details manually.
5. Ask AI to identify improvements to your literature review.

The applications generally go towards teaching you how to write better and summarize short-comings and critiques rather than writing for you. The rule of thumb here is, therefore, to use AI to “Teach a man to fish, not to feed him for a day“. This also ensures that you do not lose the ability to write academic text, but you do leverage the speed and efficiency AI has to offer.

Responsible AI Usage for Writing in 2026

To use AI responsibly for academic writing, you need to be familiar with any venue requirements, your institutions’ code of conduct, and how different AI tools works. Some general guidelines to keep in mind are:

• Don’t copy and paste AI content directly into academic work without significant paraphrasing
• Use tools like Grammarly’s plagiarism checker or Turnitin to ensure that you do not unintentionally plagiarise the work of others
• Ensure your work complies with your institution and any journal requirements (for manuscripts you intent to publish)
• Maintain your cognitive sovereignty throughout your research workflow (learn more here)

If you’re curious, you can even see what the future of academia may look like by reviewing the Stanford Institute for Human-Centered Artificial Intelligence (HAI). They have already laid the groundwork to prepare academia for the transition towards AI-generated content.

Do I need to disclose AI use in academic writing?

Yes, you should, but do not list ChatGPT as an author, as it can’t be accountable for your writing. Nature published a set of guidelines for using ChatGPT in academic work, which states the following:

[…] researchers using LLM tools should document this use in the methods or acknowledgements sections. If a paper does not include these sections, the introduction or another appropriate section can be used to document the use of the LLM.

Adding ChatGPT to your acknowledgement section does not hurt and is becoming a standard practice.

The Future of AI in Academia

AI text generation and AI detection go hand in hand, as the latter is trained on the output of the former. However, the closer AI text generation comes to human texts, the more difficult it will be to tell the two apart. With every new model, the results are increasingly human-like (In a previous article, I described how much the new o1-preview model outperforms the previous one) while human performance remains constant. It is, therefore, just a question of time until the two become indistinguishable. How will the academic landscape look if AI can write entire papers without obvious downsides?

I think this shift will profoundly transform academia. More than 500 years ago, before the printing press, the physical form of an idea, the book was valuable; today, a PDF of almost anything is free or fairly cheap. However, books only transmit ideas and require humans to “package” ideas into transmittable, written forms, making books still valuable today. AI now challenges the value of this packaging step by promising to automate it. The printing press enabled the reproduction of a book without the need for a human scribe, AI enables the creation of a book from an idea without the need for a human. In other words, AI will allow us to think in ideas, not papers and publications, and this has massive potential.

Think of an application in the near future, where instead of publishing a paper, which can take months with many authors, you release your results and ideas to your personal “scientific AI agent”. This agent is intimately familiar with your work, and other scientists also have such agents. Now, instead of publishing a paper, your agent can share the work in an electronic form with other agents and if they deem it relevant to their human, they alert the human. You AI agent can communicate the contents of this paper in any form or language you wish and contrast its relevance to your research, making the published paper obsolete and saving academics months of their time.

This is why I believe that in the not so distant future academic writing will slowly die out. It is an inefficient and slow process that has no other purpose than packaging ideas for efficient transmission to other humans, just like medieval scribes were inefficient in writing down ideas for transmission. I hope that this will leave researchers with more freedom to pursue their ideas and advance science, using their time to explore ideas faster and in more depth.

Summary

AI detectors are a common way to evaluate if a text was generated by a human or AI. However, not all AI detectors are accurate, and their reliability depends heavily on the type of text as well as the model or tool being used.

If you can follow basic guidelines for using AI in your writing, you won’t need to work to avoid AI detection, because you won’t be at risk. The goal is to use AI to enhance productivity, without compromising your cognitive sovereignty and influence in the project. Learn more about using AI while remaining at the centre of your writing process through these other articles:

👉 Andrej Karpathy’s LLM Wiki doesn’t work for Academics

👉 Concept-Based Literature Review With Obsidian and AI

👉 Jenni AI Review for Students and Researchers 2026