Money Magazine recently published research on the best colleges in America. Its methodology includes metrics that are weighted based on graduation rates, costs, financial aid, student debt, and alumni salaries. My daughter is looking at University of California schools (among other options), and I noticed that while UC Berkeley, UC Davis, UC Irvine, and UC San Diego achieved 5-star ratings, UCLA “only” received 4.5 stars.
Beyond what was stated in the methodology, I could not find detailed information about the raw scores that led to the star ratings, so I decided to consult various large language models (LLMs) to see if they could do some digging for me and propose plausible explanations for why UCLA wasn’t given a full 5 stars as I would have expected.
I asked the exact same question to Copilot, ChatGPT, and Claude: “Money Magazine released their 2026 ratings of the best colleges in the US (https://money.com/best-colleges/). Try your best to find out why UC Berkeley, UC Davis, UC Irvine, and UC San Diego achieved 5-star ratings while UCLA only received 4.5 stars. Make sure to examing their scoring methodology and determine what criteria may have caused UCLA to not achieve a full 5 stars.” Note that I copied and pasted the same typo (“examing” should have been “examine”).
Copilot didn’t spend much time researching this topic, and it started spitting out an answer 1-2 seconds after I pressed the “return” key. One of its main explanations for UCLA not achieving 5 stars was due to affordability which didn’t make sense based on the table above, so I followed up with a 2nd prompt: “UC Berkeley has a higher estimated full price and estimated priced with average aid than UCLA, yet UC Berkeley achieved 5 stars. Are you still confident in your assessment? If not sure, then state so.” Copilot then backtracked and revised its explanation, and its revision sounded more plausible to me. Here is
Copilot’s full response.
A similar thing happened with ChatGPT which spent only a second or two longer to “think” than Copilot, and it responded almost immediately. It too explained that cost was a factor in it not achieving 5 stars, so I followed up with the exact same 2nd prompt: “UC Berkeley has a higher estimated full price and estimated priced with average aid than UCLA, yet UC Berkeley achieved 5 stars. Are you still confident in your assessment? If not sure, then state so.” Like Copilot, ChatGPT also backtracked and revised its explanation. Here is
ChatGPT’s full response.
On the other hand, Claude thought long and hard about its response. It displayed multiple websites that it was consulting to formulate its response, including Money Magazine’s methodology page and the websites from each of the schools mentioned. I can no longer see the pages it was consulting, as they disappeared and were not displaying in the final response. Although I didn’t use a stopwatch, I’d estimate that Claude took 20-30 seconds to think before providing a response. I felt that Claude’s explanation was the most analytical and credible of the 3 LLMs, and unlike Copilot and ChatGPT, I didn’t feel that any of its responses were contradicted by the data. Here is
Claude’s full response.
Of course, these are my personal opinions which are completely subjective. Also, my conclusion that Claude performed the best on this specific task are not necessarily generalizable to conversations about other subjects. The take home message is that if you often consult LLMs, I’d encourage you to always think critically about LLM replies and to explore multiple LLMs to compare and contrast their responses and determine which ones are best suited to your needs.
