U.S. political campaigns inundated with deceptive AI-generated ads ahead of midterm elections
AI-generated content also proliferated during the 2024 presidential election
"Regulation alone cannot stop it": Growing influence of trust capital and media literacy

Artificial intelligence (AI) technology is emerging as a new variable in U.S. politics. As synthetic campaign ads that realistically reproduce candidates’ faces and voices appear with increasing frequency, the risk of distorting voters’ judgment has intensified substantially. State governments and Congress are moving to establish institutional safeguards aimed at minimizing disruption, but experts warn that regulation alone cannot stem the proliferation of deceptive content, given the rapid advancement of AI generation and editing technologies and concerns surrounding freedom of expression.

AI Advertising Battle Engulfs U.S. Politics

Axios reported on September 9 that “a succession of synthetic advertisements using AI to replicate the appearance and voice of opposing candidates has ignited controversy in U.S. politics.” In one instance, Ken Paxton, a Republican candidate for the U.S. Senate in Texas, featured an AI-generated version of his Democratic rival James Talarico in a campaign advertisement. The AI-generated Talarico delivered decontextualized excerpts from his actual remarks on biological sex and religion, with some of the language altered. In June, an AI-generated video depicting Talarico in an outfit resembling costumes from the film “The Sound of Music” while singing in praise of transgender children was also produced and widely circulated.

Similar cases have surfaced in other electoral districts. During the Republican primary held in May for Kentucky’s 4th Congressional District, an attack ad used AI to depict incumbent Representative Thomas Massie dining with Democratic Representatives Ilhan Omar and Alexandria Ocasio-Cortez (AOC), checking into a hotel and holding hands with them. Massie’s campaign retaliated with an AI-generated advertisement showing his rival Ed Gallrein abandoning President Donald Trump on a battlefield. In Maine, a political action committee (PAC) supporting Republican gubernatorial candidate Jonathan Bush aired an AI-generated advertisement portraying rival candidate Robert Charles carrying a bag of cash alongside former President Barack Obama.

Similar Cases Proliferated During the Previous Presidential Election

Controversy over AI-based political content also erupted during the 2024 U.S. presidential election. The most prominent example was the “Biden deepfake robocall” incident ahead of the New Hampshire Democratic primary in January 2024. An automated call imitating President Joe Biden’s voice with AI sent approximately 20,000 voters a message urging them not to vote in the primary. In August of the same year, President Donald Trump shared an AI-manipulated image on social media that falsely suggested pop star Taylor Swift had endorsed him. An image of Canadian actor Ryan Reynolds wearing a T-shirt bearing the name of former Vice President Kamala Harris, then the Democratic presidential nominee, also spread online, but it too was an AI-generated fabrication.

The scale of the problem was also evident in quantitative data. In 2024, the News Literacy Project (NLP), a nonpartisan U.S. civil society organization, disclosed 576 instances of manipulated content, stating that “social media posts containing false claims and fabricated endorsements had surged ahead of the November election.” Images distorting candidates’ appearances or reputations accounted for the largest share, at 240 cases, or 42% of the total. These were followed by 100 images featuring fabricated candidate endorsements (17%), 91 propagating baseless conspiracy theories (16%), 74 containing false information about the electoral system (13%) and 71 misrepresenting candidates’ policies and campaign pledges (12%).

Efforts to Erect Regulatory Barriers

As AI emerges as a new variable in the electoral process, U.S. political institutions are accelerating their regulatory response. According to the National Conference of State Legislatures (NCSL), 31 states had enacted laws regulating deepfakes used in electoral and political messaging as of June. Minnesota and Texas restrict the distribution of political deepfakes that falsely portray candidates during designated periods immediately preceding an election, while Maryland prohibits deceptive election-related deepfakes throughout the year. Most other states have designed their regulations around mandatory disclosures indicating when images, audio or video used in advertisements have been generated or manipulated with AI.

Congress is also pursuing legislation directly targeting AI-generated political advertising. The “AI Transparency in Elections Act of 2026,” introduced by Democratic Representative Joe Morelle on July 23, would require political advertisements to include clear disclosures when their images, audio or video have been substantially produced using generative AI. Separate legislation intended to regulate AI-enabled candidate impersonation and fraudulent fundraising is also advancing. Democratic Senator Adam Schiff introduced the “AI Ads Act” in the Senate on July 27, while Democratic Representative Ro Khanna introduced the corresponding bill in the House on July 30. The legislation would add provisions governing generative AI to the existing Federal Election Campaign Act. Specifically, it would explicitly bring within the scope of existing prohibitions on “fraudulent misrepresentation” the use of AI to reproduce a candidate’s face or voice in ways that falsely imply the candidate approved an advertisement or solicited donations.

Table 1. Limitations of Regulations on Deceptive AI-Generated Content

Clear Limits to Institutional Safeguards

Market observers nevertheless argue that institutional regulation alone cannot fundamentally arrest the proliferation of deceptive AI-generated content. Digital images, audio and video ultimately consist of data such as pixels and audio samples, allowing generative AI to replace or reconstruct individual elements with relative ease. More recently, access to a candidate’s photographs and voice recordings has become sufficient to create realistic synthetic content without specialized technical expertise, while open-source models can circumvent even the generation restrictions imposed by developers. Measures such as Google’s “SynthID” and the Coalition for Content Provenance and Authenticity (C2PA)-based “Content Credentials,” which record content provenance, have been proposed as potential safeguards. Yet not every generative AI company or platform has adopted these technologies. Moreover, information stored as metadata may disappear when content is uploaded to social media and can be damaged when an original file is captured or reprocessed.

AI detection technology likewise offers no comprehensive solution. Deepfake detection tools currently in use rely on models trained on specific datasets to estimate the probability that a piece of content is synthetic. Their accuracy inevitably deteriorates whenever a new generation model or editing method emerges. Tests of publicly available AI detection tools conducted by the Reuters Institute for the Study of Journalism found that deepfake videos could evade detection through simple edits such as reducing video quality or trimming selected segments. Legal constraints also make it difficult to expand the scope of regulation indiscriminately. Political expression in the United States receives robust protection under the First Amendment. Broadly prohibiting political content solely because it was generated using AI could therefore invite constitutional challenges.

Trust Determines the Quality of Information

Experts identify trust in the media as the key to resolving this confusion. As technological advances make it increasingly difficult to distinguish authentic content from fabrications, users are more likely to assess the credibility of the entities that produced and verified the information than the content itself. News organizations with a sustained record of accurate reporting under consistent verification procedures can serve as benchmarks for determining the veracity of information. Trust accumulated over an extended period consequently functions as a form of “trust capital.” The more clearly news organizations identify their sources and evidence and the more transparently they disclose their fact-checking procedures, the greater the value users assign to the information those outlets provide.

Such trust capital influences not merely reputation but the entire architecture through which information circulates. If users give priority to highly credible sources and platforms likewise afford greater visibility to content from verified information providers, less credible information will progressively lose its reach. Under this model, false information is displaced through the repeated choices of users and platforms rather than through blocking or deletion alone. The relative importance of media literacy also increases in this process. The information market’s filtering function can operate effectively only when users are capable of independently examining a piece of content’s provenance, author, supporting evidence and coverage by other credible media outlets.