Travel Planning With AI: I Tested It for a City I Know Inside and Out

Would you let AI plan your next vacation? For the time-poor among us, it's a tempting proposition.

There's no two ways about it -- planning a great trip is a labor-intensive activity. If you're not careful, you can spend as long reading about and preparing to visit your chosen destination as you'll eventually spend on the ground there. And with many major tourist cities attracting more visitors than ever, even the most seat-of-their-pants kinds of traveler needs to do some preparation.

Over the past year, generative AI (in which you use a prompt to ask an algorithm to generate content -- either text, image or video) has become a publicly available plaything, leading it to be touted as a promising, if often over-hyped, tool to be used in almost any context you can think of. No surprise then that as the world has experimented with OpenAI's ChatGPT tool, many people have been using it to quickly and efficiently create travel itineraries.

A number of travel-specific AI services have sprung up as a result, offering to take on the burden of generating itineraries. But are they any good? Results have been mixed, suggesting there's no black-and-white answer to this question. It's not so much whether you should use AI to plan an itinerary, but how to use it to achieve the best results. 

As someone who lists trip planning as a top-three hobby, I was keen to explore its potential and its limits.

I decided the best methodology would be to use my own city of Edinburgh as a testbed for a three-day trip. I know Edinburgh like the back of my hand, it's a popular destination and it's the ideal location for a quick getaway, even from the US. It's small enough that you won't leave feeling as though you've barely scratched the surface, but big enough and packed with history and tourist attractions so that there's no chance of you finding yourself at a loose end by Sunday afternoon.

I used a range of AI travel tools to build itineraries including ChatGPT, GuideGeek, Roam Around, Wonderplan, Tripnotes and the Out of Office, or OOO, app. Here's how I got on.

Travel itineraries: An art, not a science

A well-constructed travel itinerary is more than just a list of attractions, neighborhoods and local businesses. The best itineraries will string your day together in a way that makes sense geographically and thematically. 

The journey between attraction A and attraction B will be part of the fun, taking you down a picturesque street or providing a surprising view you might not otherwise have seen. It will also be well paced, taking into account that by the third gallery of the day, even the most cultured among us will likely be struggling with museum fatigue.

Very few of the itineraries I asked AI to create for Edinburgh fit this brief. Roam Around, Tripnotes, OOO and Wonderplan clearly had no sense of the geography of the city. ChatGPT and GuideGeek did a much better job of grouping attractions according to neighborhood, building a day around Edinburgh's Old Town, a day around Holyrood and Arthur's Seat, and a day around the New Town area.

See also: Our sister site ZDNet's ranking of the best travel planner apps

The most egregious examples of nonsensical itinerary planning came from OOO and Roam Around when I tested them further by asking them to create larger itineraries for a seven-day trip around Scotland. Roam Around presented me with a fantastic selection of activities and attractions around the country but suggested I drive hours back and forth to Edinburgh for every meal. 

Meanwhile, Day 1 from OOO started me off with breakfast and a graveyard in Edinburgh, a Glasgow Museum at lunchtime, Glenfinnan Viaduct in the Highlands in the afternoon, before staying overnight at a hotel on the Isle of Skye (but with drinks and dinner back in Edinburgh). That's 10 and a half hours of driving, and it only got more exhausting from there.

I approved of the majority of the suggestions AI came up with for Edinburgh, and for Scotland more broadly, both in terms of attractions and local businesses. But aside from ChatGPT and GuideGeek, which were able to create thematically and geographically cohesive itineraries when pushed, the way they were strung together was often counterintuitive.

It's clear that many AI tools are doing little more than aggregating information found elsewhere -- an approach that leaves you with what Lonely Planet Senior Editor Laura Motta describes as "Google Search scrambled eggs." 

Lonely Planet, a sister site to CNET, is currently building its own AI tool, Motta tells me, and is attempting to go beyond simply providing you a jumbled list of must-see places. Instead, it's using preexisting, expertly sourced granular knowledge of destinations to help people build itineraries around must-see attractions.

"Of course you want to see the Eiffel Tower -- maybe don't want to skip that if it's your first time to Paris," Motta says. But the plan is to introduce you to other things in the vicinity of these main sites that you might otherwise have missed -- a bakery perhaps, or a nearby square containing an important piece of history. 

"It's about a balance between seeing the big sites and getting out and doing other things," Motta says. "And AI, at this point, is not great at recommending the other things."

Verify, crosscheck and double source

You're likely going to come to Edinburgh (or any city) with a list of sites you absolutely want to see, and any good AI itinerary will help you find these. But not all. When I asked Tripnotes what I should see with a long weekend in my city, the two activities it suggested for the first day were the Fruitmarket Gallery (good, but not exactly a must-see) and a hostel.

I have so many questions, but mainly, where did it get these ideas from? There's not really a way of knowing, which means it's up to you to do some digging of your own. "It's going to be really important to validate some of what comes out, because the recommendations are not sourced particularly well," Motta says.

Even when AI recommends the big attractions, you're not totally safe. Every AI itinerary I generated for Edinburgh recommended a trip to Edinburgh Castle -- predictable, but fair. But did you know that tickets for Edinburgh Castle regularly sell out days in advance? And that if it's too windy, the castle will close -- often at short notice?

Real-time and even recent information are just a couple of things you shouldn't currently expect to find on your AI travel itinerary. The fact that AI uses historical data makes it incredibly backward looking, which Motta identifies as the biggest issue when it comes to using it for travel planning right now. "The guidance is never going to be of the moment the way that you need real-time travel advice to be," she says.

It's going to fall to you to double-check what AI tells you -- and that goes for everything it tells you.

It's particularly important that you don't follow food and drink recommendations blindly. Almost every service I used pointed me to a cafe, bar or restaurant I know to be permanently closed. Additionally, Roam Around suggested I have breakfast at a gastro pub one morning and a pizzeria the following morning -- neither of which opens before midday.

The last few years in particular have seen a huge shakeup in the food and drink industry in cities around the world, with some businesses closing due to the pandemic, others opening and yet more popping up or reopening elsewhere. "There's going to be a lot of inaccuracy in there just because of what's happened in the world," Motta says.

If you're the kind of traveler that likes to visit the hottest new openings in town, you're going to need to resort entirely to your own research. At this time, the AI travel tools available are simply not capable of staying on top of trends -- and they're honest about it.

When I asked ChatGPT about the most exciting new openings in Edinburgh, it responded letting me know that its knowledge is only current up to 2021. It then pointed me in the direction of TripAdvisor, Yelp and local food and travel publications. (OpenAI announced last week that the new GPT-4 Turbo, the brains behind ChatGPT, is trained on more up-to-date data, up to April 2023.)

The promise of AI is that it can take away the heavy lifting of creating a travel itinerary. The reality, at least for now, is more complicated. AI can provide a great framework for your trip, but it's still up to you to fact-check it before traveling -- so don't ditch the guidebooks, Google Reviews and TikToks filmed in low-lit restaurants just yet.

Helping AI to help you

With generative AI as with so many things in life, you get out what you put in. Asking smart questions can make all the difference to the quality of guidance you will get for your trip.

One common activity suggested by several of the AI tools I used to generate itineraries for Edinburgh was climbing Arthur's Seat -- an extinct volcano that sits right in the heart of the city. I fully endorse its inclusion in any Edinburgh itinerary (it's a unique attraction and the views are magnificent), but where I do take issue is the lack of guidance the AI tools provided when it came to the climb.

The hike up Arthur's Seat is short but strenuous with some scrambling at the top. It's achievable for most people with a good level of fitness, but not for everyone -- and definitely not without appropriate footwear. Several times a year people need to be airlifted off the hillside, and there are many weather conditions I wouldn't personally attempt it in. I certainly wouldn't want to spend the rest of the day afterward, as GuideGeek suggested doing, taking another long walk around a completely different area of the city. Likewise, I wouldn't want to make it the very last thing I did on a day in November, when the sun sets around 4 p.m., as suggested by Wonderplan.

None of the AI tools I used to generate itineraries proactively told me any of this information, but when I asked GuideGeek whether it was safe to climb Arthur's Seat in winter, it gave me great safety advice, which included warnings about the short length of daylight hours in Scotland in winter.

This isn't the only time GuideGeek's chat function served me well. All of the AI tools I used showed me a very similar list of "top" restaurants in Edinburgh to choose from during my trip. Most of these restaurants had Michelin stars, would be considered fine dining or are otherside very expensive -- not accessible options for many travelers. Noting this, I was then able to ask for much more specific advice about small independent restaurants, wine bars and affordable options. The resulting recommendations included many of my favorite establishments from around the city.

These examples demonstrate the importance of asking the right questions. AI does potentially have all of the answers you need, but you might have to dig deep to get them. Taking the time to thoughtfully structure your AI prompts will likely yield more relevant results. Queries you might want to put to any AI itinerary planning tools include asking whether you need to prebook tickets for attractions it recommends, or to help you plan visits to popular tourist sites at quieter times.

Personalization: The next frontier

Not all tools currently available have the capacity to engage in back and forth like chatbots such as ChatGPT and GuideGeek. This does give these services the edge when it comes to itinerary planning, even if it does result in some farcical conversation (such as GuideGeek telling me it was "a fellow fan of the wizarding world" when I asked for Harry Potter recommendations).

Right now, most AI tools fall under the umbrella of generative AI, in that they can generate content -- travel itineraries, for example. But we're also quickly moving toward the adoption of more interactive AI, whereby tools and services are built for us to engage with them through questions and prompts. 

Developments in interactive AI, including more natural conversation skills and the ability of AI tools and companions to get to know our personal preferences could take itinerary planning to the next level. Instead of presenting everyone with roughly the same information, this iteration of AI is much more likely to be able to learn to adapt to your personal preferences.

Travelers come in many different flavors, all with their own preferred approaches to trip planning and their own ideas about what makes a dream vacation. The most underwhelming thing about the Edinburgh itineraries I generated using AI were that they were all fairly generic. ChatGPT and GuideGeek did a fairly decent job of adjusting itineraries when asked to include more nature, more culture or more of a focus on Scotland's whisky heritage, but it felt like an off-the-shelf solution rather than a carefully curated journey tailored to my travel style. 

Even when I was presented with a solid itinerary, I still wanted the option to add in my own finds or recommendations from friends -- and some services do allow this. The way some of the travel tools -- OOO in particular -- presented the itineraries using imagery made them visually appealing, and they also left room to manually add in or adjust the trip that they generated for me. 

The plan for Lonely Planet's AI tool, Motta says, is to ensure that people still have a sense of agency when it comes to planning. There's no expectation they'll follow every recommendation. Rather, they'll pick and choose from well-sourced options surfaced by the tool and fill in the rest themselves.

"Do we want to save you time and energy when it comes to planning your trip?" Motta asked. "Yes, absolutely. But I also don't think we want to take the joy out of that process either."

AI tools have some way to go before they're able to read us as well as we can read them. No doubt there will be a point in the future when AI knows our accessibility needs, our penchant for an afternoon nap, our walking speed and our preference for a particular style of wine or art, and can plan for us accordingly.

Just because we're not there yet doesn't mean you shouldn't use AI to help plan your trip -- just so long as you remember it's there to supplement your own research, not serve as a replacement.

Editors' note: CNET is using an AI engine to help create some stories. For more, see this post.