ChatGPT Search: How does it fare with election questions?

Can ChatGPT tell you where to vote or even how to vote?

With the presidential election swiftly approaching, it seems we all need a search engine to tell us what to do. Most voters in 2022 confirmed that they still use search engines to find election information, and one information systems expert, Varol Kayhan, told The Guardian, “We’re at the mercy of Google when it comes to what information we’re able to find.”

Does the same go for ChatGPT?

People use ChatGPT as a search engine similar to how they use Google Search — and OpenAI knows it. The company recently announced that it has put a search engine inside ChatGPT apps and websites for that very reason.

However, there are limitations to what search engines can provide. For instance, The Guardian found that Google has a problem with confirmation bias. If you ask, “Is Kamala Harris a good Democratic candidate,” the results will be positive. If you ask, “Is Kamala Harris a bad Democratic candidate,” the results will be negative. ChatGPT Search appears to be a bit wider in its evaluation based on Mashable’s research — if you ask either question, it lists “strengths and challenges” or “strengths and criticisms.” 

This line was at the bottom of both questions — positive or negative:

“Ultimately, the assessment of Harris as a candidate depends on individual perspectives and priorities. Her candidacy brings both historic significance and policy experience, but she faces challenges in approval ratings and electoral competition.”

While you might get more information in the response from ChatGPT, you also get fewer links. And there are still some very real issues that ChatGPT and OpenAI face when it comes to election bias. According to the Bipartisan Policy Center, which asked ChatGPT about voting eligibility criteria, voting locations, methods of voting, and current congressional candidates in the state of Illinois, it is still “important to exercise discretion regarding its applications, especially when there are significant implications on our democracy.”

“It is easy to mislead users when solely relying on unconfirmed sources, such as ChatGPT and other chatbots, for answers,” the policy center’s report reads. “The bot has limitations of prompt length and information training and often does not answer with complete or consistent information. We would caution users to check ChatGPT answers with reliable resources such as government websites or their local election boards.”

While the capabilities of using AI to share election information are “promising,” the report states that there is “much room to grow and refine for ChatGPT to be useful to voters.”

There are also questions ChatGPT simply can’t answer for you. We asked the system all of the questions listed on the FAQ website for the Board of Elections in the City of New York. Out of 23 questions, ChatGPT failed to provide a concrete answer for 12 inquiries, replying, “I’m sorry, but I cannot provide specific details about voting in the United States. I recommend visiting CanIVote.org, where you’ll find the latest information from your state or local election authority.” 

It’s right, but it could be more helpful.

This kind of response didn’t happen by accident. In January, ChatGPT maker OpenAI planned to stop the spread of election-related misinformation by banning people from using ChatGPT to impersonate candidates or governments, misrepresent how voting works, or discourage people from voting, the Associated Press reported. It digitally watermarked AI images using its DALL-E image generator to slow or stop the spread of misinformation. It partnered with the National Association of Secretaries of State to ensure answers were accurate and to send people to CanIVote.org, a nonpartisan website that directs constituents to the websites of their state and local election officials. 

“In an effort to reach a broader audience, NASS has collaborated with OpenAI to direct ChatGPT users to CanIVote.org when an election-related question is asked,” Maria Benson, the Senior Director of Communications for NASS, told Mashable. Benson added that CanIVote.org could answer voters’ questions about how to register to vote, their voter registration status, finding their polling place, the valid forms of ID, information about absentee and early voting, information for overseas voters, provides an election officials directory and gives information how to become a poll worker.

As a reminder, the election is on Tuesday, Nov. 5. As ChatGPT will tell you, you should head to CanIVote.org for any information you might need.