Loading…
13
I ran my old chatbot against the new one for a week and the difference was clear
I set up a simple test with 50 customer service questions from my small online store. The old bot I built last year with basic rules got 32 right, but it kept giving the same wrong answer about shipping times to Canada. The new one I tried, which uses a smarter model, got 48 right and even figured out a weird question about a broken zipper. It just understood what people meant better. Has anyone else seen this kind of jump when they switched to a newer AI system?
4 comments
Log in to join the discussion
Log In4 Comments
river_gonzalez665mo ago
Hold up, you only tested 50 questions. That old bot might have nailed the shipping answer if you'd asked it differently, like saying "ship to Toronto" instead of "Canada." The new one is just guessing better on the weird stuff, but it could totally fail on a simple question the old one had locked down.
5
colej746mo ago
Yeah that jump from 32 to 48 is huge lol. The new ones just get context way better, it's wild.
1
noah3876mo ago
Tell colej74 it's like the old models were reading each word alone and the new ones see the whole sentence... makes a world of difference. You really notice it when you ask for something with a few steps. Honestly makes the older ones feel broken now.
6