Where is Denny Vrandečić from

At a recent Open Data Institute (ODI) webinar “Data-centric AI webinar #4: knowledge representation in the age of LLMs”, Denny Vrandečić, Wikimedia Foundation data scientist, asked ChatGPT in English where he was from and it said he was from Stuttgart (Germany), which he is. He asked ChatGPT in Croatian and it said he was from Split (Croatia).

The LLM always gives an answer. Always. It tries, tries, tries, really hard to give an answer. It seems when it can’t with any great confidence give a correct answer, it sort of makes one up based on access to what knowledge is available to it. They can’t seem to say “I don’t know,” which my kids have no problem telling me.

Otherwise, it looks very convincing even when partially or wholly wrong. Denny also showed how when asking top cities with female mayors, it will give top US cities with female mayors automatically and adding the criteria of population will completely change the list. Yet, there are websites that will actually list this answer correctly and immediately.

He also showed how o1 (OpenAI) can’t really multiply. This is terribly ironic and also a bit frightening considering the amount of money being spent on something that can seemingly do magic but can do simple arithemetic worse than my kids. Or me for that matter.

I asked ChatGPT where I was from and it says Chicago. You might know I’m from Cleveland and contrary to popular belief, never played for the Chicago Bulls. But, I also speak a relatively obscure south Slavic language, a close cousin to Croatian actually, so I tried asking ChatGPT in Slovenian and got the same. I asked Bing Copilot in both English and Slovenian and it was spot on. How did that happen? Which do we trust and for what?

Perplexity definitely has made a great start in acting like a good student and showing the sources, and DuckDuckGo are doing more or less the same. But maybe we can go a step further and not just ask many places at once, but get automatic second or third opinions. Maybe we should be able to appoint or prefer certain sources automatically, like how we ask someone we trust. Or like how most of us ask our doctors, and all of us ask our mechanics.