Multimodal-Ai

Back in May I wrote about asking ChatGPT and Claude where to put my circulator fan in my study. Last week the question was less relaxed and burning. The UK broke its June temperature record three days running, my study is a small room with the sun aimed straight at it, and I went back to both models to see if they could help. What came back read well and annoyingly didn’t work. Both kept telling me to aim the air just above my head, which puts the column over you, and when I said it wasn’t working they’d tell me to nudge it up a little more and that I was nearly there, pointing me the whole time at a placement that wasn’t going to achieve what they calculated. ...

The biggest change in chat models over the last two years is not the one most articles focus on. It is that they can now actually look at things. Image input went from a feature you would test once and forget about to something I use weekly without thinking about it, and the gap between describing a problem in words and just showing the model what you are looking at turns out to be much bigger than I expected. The first time it really landed for me was a few years ago when I started feeding ChatGPT photos of error screens and bits of hardware I could not be bothered to describe. By the time photo-based questions felt routine I had built up enough trust in ChatGPT specifically that vision tasks became one of the things I would default to it for, even as Claude took over for almost everything else I do. ...