Today I was debugging an integration between two systems. Transactions weren’t being found, cause unknown.

Classic black box: data goes in somewhere, something breaks somewhere, but where exactly — not obvious.

In cases like this the only real approach is to collect full information about good transactions and bad ones across all systems, compare them, and look for the trigger.

The problem is the trigger might not be where you think. A connection issue, an edge case the rules don’t cover, or someone manually entered an order with a typo — and it blew up five steps later, somewhere completely different. So where do you even look?

Brought in AI to speed things up. Gemini has a large context window, it’s helped me with this kind of analysis before.

Gemini’s conclusion: the cause is order statuses in the CRM.

Lol. A status is a consequence, not a cause. First the transaction stalls, then it gets the status “on hold.” The model saw a correlation and called it a cause. Noise instead of signal.

Tried GPT with the same data, added a constraint about statuses in the prompt — got hypotheses I could actually test. Found patterns that would’ve taken me hours to find manually.

AI is great at speeding up initial analysis, but it can get cause and effect wrong. If you don’t understand the context yourself, it’ll confidently lead you in the wrong direction.