During a recent press appearance, OpenAI CEO Sam Altman said that he’s observed the “IQ” of AI rapidly improve over the past several years.
“Very roughly, it feels to me like — this is not scientifically accurate, this is just a vibe or spiritual answer — every year we move one standard deviation of IQ,” Altman said.
Altman isn’t the first to use IQ, an estimation of a person’s intelligence, as a benchmark for AI progress. AI influencers on social media have given models IQ tests and ranked the results.
But many experts say that IQ is a poor measure of a model’s capabilities — and a misleading one.
“It can be very tempting to use the same measures we use for humans to describe capabilities or progress, but this is like comparing apples…