# AI wrote the tests: 7 signs they check nothing

Source: https://checkcraft.ru/en/guides/ai-tests-empty-check.html · updated 2026-11-20

Tests from a neural network are almost always green. Seven signs of an empty test and a five-minute check: run the test against a deliberately wrong response.

Practical workflow

Checkcraft is a desktop app for checking APIs and logs without code, with a free edition. [Download, no sign-up →](https://checkcraft.ru/en/download-checkcraft.html#free-download)

The agent wrote twenty tests and all are green. That does not mean they check anything: a green test that cannot fail is worse than no test, because it builds confidence. Here are seven signs of an empty test and a way to check one in five minutes.

Checkcraft · 20 November 2026

Get help with your first test →

## Seven signs of an empty test

1. Only status 200 is checked. 2. The expected value was copied from the actual response, not from the requirements. 3. The test checks that a field exists, not that its value is right. 4. A try/except swallows the error, or the assertion sits inside a condition that never runs. 5. The test checks a mock instead of the real service. 6. After a failure the agent "fixed" the test by changing the expectation. 7. The test has never failed: nobody ran it on wrong data.

## A five-minute check

Take any test and feed it a deliberately wrong response: a different total, an extra field, a wrong order status. If the test stays green, it checks nothing. The practice API, where every response is 200 and the bugs are planted on purpose, is made for this: a proper total test fails on order 5001, because 3170 − 300 = 2870 while the response says 2970.

```
# An empty test: passes on any response with status 200
def test_order():
    r = requests.get(URL)
    assert r.status_code == 200

# A test that checks something: the total by the rule from the requirements
def test_order_total():
    order = requests.get(URL).json()
    assert order["total"] == order["subtotal"] - order["discount"]

# Try it on the practice order: the second test fails (2970 instead of 2870)
URL = "https://checkcraft.ru/practice-api/v1/orders/5001.json"
```

## How to avoid empty tests

Take expected values from the requirements and write them down before you see the response. Run every new test at least once on wrong data. And split the roles: the agent writes the checks, and an app runs them from saved rules. Then the agent has nowhere to bend the expectation, and every change is visible. That is how Checkcraft works with AI agents over MCP.

Next step

## Repeat this on your own API

Checkcraft is a desktop app that saves API response checks without code and shows what did not match. Start with a sample, then use your own sanitized data. Free runs on Windows and Linux with no time limit.

[Download Free](https://checkcraft.ru/en/download-checkcraft.html#free-download) · [Download and first run](https://checkcraft.ru/en/download-checkcraft.html)

No sign-up or email needed, the archive is ready in a couple of minutes. See the [edition limits](https://checkcraft.ru/en/pricing.html).

## Continue with a related task

[Practice API with bugs: find all 10](https://checkcraft.ru/en/guides/api-testing-practice.html) · [Checkcraft MCP server: an AI agent checks the API, logs and database](https://checkcraft.ru/en/guides/mcp-server.html) · [HTTP 200 but the result is wrong](https://checkcraft.ru/en/guides/http-200-wrong-result.html)

Start with the linked example and compare its result with your own test requirement.

Ask a person

## Get help with your first test

Tell us what you want to test or where you got stuck. We can help you start Checkcraft, choose an example and understand your first run. You can ask before downloading.

[Ask in Telegram](https://t.me/checkcraftchat)Send an email

The Telegram group is public: do not post passwords, tokens or confidential work data. A description of your task is enough to start.

You can also email us support@checkcraft.ru

Page card: /ai/pages/en/guides/ai-tests-empty-check.md
