ChatGPT 3.5 Review: First Doesn't Mean Best

6.0/ 10
SCORE

ChatGPT 3.5

Pros

• Fast
• Decent accuracy

Cons

• Prone to hallucinate on more complex queries
• Poor job of citing sources
• Leaves it up to user to cross-reference Google

Basic info:

• Price: Free
• Availability: Web or mobile app
• Features: Voice recognition
• Image generation: No

ChatGPT is mind-blowing, but not without faults. When OpenAI's chatbot stormed onto the internet in late 2022, it dazzled with its ability to answer seemingly any question with a unique answer. 

This type of power, being able to ask open-ended questions and get back useful answers, is not how traditional internet search works. Putting questions into Google yields links to various articles and Reddit threads with people giving their research and opinions. It's up to you, the end user, to absorb all that information and synthesize a conclusion in your head. 

ChatGPT does that synthesis for you. It truly does feel like a calculator, but not for numbers, instead, for the wealth of human knowledge found online. 

There are different tiers to ChatGPT. The one available for free, version 3.5, uses 175 billion parameters. Think of parameters as the number of pieces of information. The more parameters a model has, the better it can understand language and produce nuanced sentences. While that certainly is a lot, it pales in comparison to ChatGPT 4.0, which reportedly has 1 trillion parameters, but costs money based on the number of inputs and outputs. For this review, I tested the free version.

How CNET tests AI chatbots

CNET takes a practical approach to reviewing AI chatbots. Our goal is to determine how good it is relative to the competition and which purposes it serves best. To do that, we give the AI prompts based on real-world use cases, such as finding and modifying recipes, researching travel or writing emails. We score the chatbots on a 10-point scale that considers factors such as accuracy, creativity of responses, number of hallucinations and response speed. See our page on how we test AI for more.

When using ChatGPT, keep in mind that the service automatically collects the information you put into its system, so be mindful of giving the service any personal information. For more information, see OpenAI's privacy policy.

Shopping

It's hard to recommend ChatGPT 3.5 as a shopping aid. Because its training data is only up until September 2021, it lacks information about newly released products. It's hard to ask it to make comparisons on the latest TVs and cars if ChatGPT 3.5 has nothing to work with. 

Google Gemini and Perplexity are all connected to the open internet, meaning that information on new products is present. Microsoft Copilot was recently updated to incorporate GPT-4 Turbo, a more advanced AI model from OpenAI that has training data up until April 2023. Like Gemini and Perplexity, it too is linked to the open internet to bring in links and other more recent bits of information. Anthropic's Claude isn't connected to the open internet, but its training data goes up until August 2023, so at least it's a bit more recent. 

Recipes

Searching for recipes on Google can be a slog. Finding a good recipe sometimes means having to scroll through paragraphs of needless backstory to get to the recipe itself. 

ChatGPT cuts out all that fluff and gives you a recipe, instantly, while also allowing you to add additional variables specific to your dietary needs. These types of recipes might not be as easily available via a Google search.

For example, I asked ChatGPT for a chicken tikka marinade recipe and it produced one in seconds, minus a backstory of visiting an Indian restaurant for the first time on Curry Mile in London. When I asked it to make a vegetarian version, ChatGPT remixed it instantly, swapping the chicken for paneer, an Indian cottage cheese. 

At the same time, ChatGPT 3.5's recipe generation lacked context. While ChatGPT does get straight to the point, the author of this other recipe I found pointed to the importance of using Kashmiri chili powder, kasuri methi (dried fenugreek), chaat masala, amchur (dried mango powder) and black salt. ChatGPT's version of this recipe was noticeably barebones by comparison.

When asking Gemini this same question, it was able to include ingredients like Kashmiri chili powder and amchur. I guess there's some secret sauce Google is using to get a bit more culture out of its recipe generation. In our tests, Gemini performed the best, followed by Microsoft Copilot (in creative mode), then ChatGPT 3.5, and it was a tie between Perplexity and Claude. However, no AI chatbot excelled at this test.

Research and accuracy

Research can be a slog of jumping between Google searches, research papers and public libraries. Generative AI can condense all of that hard work, absorbing the wealth of published knowledge online and helping synthesize information and giving specific answers to specific questions. 

It's also handy if generative AI can pull up the sources it's referencing. However, ChatGPT 3.5 doesn't source much at all. 

Whatever information ChatGPT dumps on your screen requires having to go to Google to actually find it and link to it, which ChatGPT makes challenging. Often, when asking ChatGPT for an exact source, it'll say that as an AI language model, it doesn't have direct access to real-time data or the ability to browse the internet. The inability to easily cross-reference sources makes the actual real-world usefulness of ChatGPT questionable. Sure, among friends, you may cite ChatGPT and get away with it. But for school or work, you'll be left scrambling on Google, potentially on a futile chase looking for a source that might not even exist.

It seems that OpenAI has tweaked ChatGPT to often not point to specific papers or sources when asked. This could have been because in the past it would make up papers that didn't exist. One user on Reddit from 10 months ago said that when asking, "Give me some papers on the relationship between homeschooling and neuroplasticity," ChatGPT would point to a paper that didn't exist. Asking that question now, ChatGPT says as an AI model, it can't browse the internet or access specific papers. However, when asking questions regarding the neurological implications of COVID-19 on health, ChatGPT immediately pointed to four sources, all of which were easily found on Google. 

It doesn't help that ChatGPT 3.5, at times, would hallucinate. Hallucinations are when an AI chatbot produces an incorrect answer but says it with confidence. It wasn't always immediately obvious and took some additional digging around on Google before I realized that ChatGPT 3.5 was making up factoids. 

This inconsistency is annoying, as it's hard to pin down why ChatGPT 3.5 bars citing some pieces of information while allowing others. 

Compared to other Chatbots, Claude performed the best at synthesizing different bits of research as well as linking to sources. Copilot, in creative mode, also performed similarly to Claude, finding the nuances in a complex topic. Google Gemini, with its access to the open internet, did a better job than ChatGPT 3.5, but hallucinated in odd ways, making up the names of studies that didn't exist. And Perplexity, while it did a decent job, worked in sources that weren't academically reliable. 

Summarizing

ChatGPT 3.5 certainly shows its limits when asked to summarize an article. I asked it to summarize an article I wrote earlier this month regarding ChatGPT's impact across the tech scene at CES 2024. I pasted the entire article into 3.5 and the summary it yielded was lacking. It picked up the background information and mentioned the main thesis, but failed to bring the point home. It also abruptly stopped its summary, ending midway through a sentence. When I asked why it had stopped summarizing, ChatGPT 3.5 apologized and gave another summary, only to abruptly stop at the same spot. 

ChatGPT has a 4,096 character limit, according to Android Authority, meaning it can't summarize a 940-word article. Essentially, don't expect it to help you parse through large legal documents or terms of service agreements.

In comparison, Gemini has the ability to summarize articles just based on a link, but its link-based summaries were barebones and rather useless. When I pasted the whole article into Gemini, however, it actually did a better job of summarizing my article than ChatGPT 3.5.

Perplexity and Claude failed to get the full scope of my article.

Travel

Looking up travel ideas for major cities like Los Angeles or Tokyo isn't hard. The internet is packed with websites, TikToks, tweets and other options showcasing the most populous cities in the world. What about Columbus, Ohio, though? That's where an AI chatbot can come in handy, filtering through TripAdvisor, Reddit and other posts to put together a serviceable itinerary.

When I asked ChatGPT 3.5 for a three-day travel plan in Columbus, it did a surprisingly solid job of putting together packed itineraries full of activities. It recommended places to see as well as restaurants to visit. And unlike Google Gemini, all the restaurants it recommended were actually real. Why Gemini was more prone to hallucinations than ChatGPT in this test isn't clear. But it does point to how much tuning OpenAI has done to ensure information remains accurate. 

According to CNET's Bella Czajkowski, who's from Columbus, the only potential ding was that ChatGPT 3.5 recommended going to the Short North Arts District both on the first and third day. Generally, people like travel plans that don't repeat locations. 

Perplexity made vague recommendations whereas Claude performed well, but had one error. Gemini hallucinated the most, making up the names of restaurants that didn't exist.

Of the chatbots, Copilot performed the best, making a clean and organized list of activities, all bullet-pointed, along with pictures and emojis. 

Writing emails

ChatGPT does well in writing basic emails. From finding an excuse for not turning in your homework on time to an apology for missing an event, ChatGPT returns results that read believable, if not always genuine. When asking it to generate an email asking your boss for time off, ChatGPT 3.5 does default to overly formal-sounding language, but asking it to lighten up the language does create something more passable. Still, it'll require some tweaking to sound believably human. Even when asked to dilute some of the formality, it can still come across as robotic. So, ChatGPT will certainly give a solid template, but will require some editing to make it sound real. 

Comparatively, Gemini wrote emails well and was easy to tune to make it sound more casual and humanlike. Perplexity performed well at writing basic emails, but faltered on more complex topics, often coming off as robotic. Claude performed the best, crafting sentences with great nuance and believability. Copilot had no problems writing basic emails, but it refused to answer prompts about more controversial topics.

ChatGPT 3.5 is fine for most people, but there are better options

For most basic queries and even a few more complex ones, ChatGPT 3.5 will get the job done for most people. The answers it yields are serviceable, and often with a bit of tweaking, is still a lot less work than writing things yourself. 

That's not to say that ChatGPT 3.5 should be used as an end-all solution. It's a tool that, when used in conjunction with Google and other resources, can help cut time down on research and discovery. Still, to get the most out of it, questions have to be worded in a manner that can get AI chatbots to respond most accurately. This is referred to as prompt engineering, a subspecialty that may become a necessary skill as AI chatbots permeate throughout the tech we use. 

While ChatGPT 3.5 is user-friendly enough so that most people can still find value in it, it's best to keep your guard up and not to take ChatGPT's answers as absolute. It's always best to do a bit of fact checking, which means that Google will still be your web browser's homepage for the time being. 

It raises the question: Why use ChatGPT 3.5 when you can use Microsoft Copilot, which uses GPT-4 Turbo, for free? Given that GPT-4 Turbo reportedly uses over 1 trillion parameters, which are values that make an AI model more accurate, and is connected to the internet, there really isn't any reason to download the ChatGPT app. 

Given its drawbacks, it's hard to recommend ChatGPT 3.5 over Copilot, Gemini, Perplexity or Claude, despite how revolutionary it was back in November 2022. 

Editor's note: CNET is using an AI engine to help create a handful of stories. Reviews of AI products like this, just like CNET's other hands-on reviews, are written by our human team of in-house experts. For more, see CNET's AI policy and how we test AI.