The AI confidence trap: why spotting deepfakes is only half the battle

The AI confidence trap: why spotting deepfakes is only half the battle.

---

As AI makes online scams increasingly convincing, Avast’s latest Safe Tech Report highlights a growing disconnect between people’s confidence in recognising fake content and their willingness to protect their personal information.

AI is changing the way we work, communicate and access information. From creating content and improving productivity to helping us solve everyday problems, AI offers significant benefits. However, the same technology is also giving cyber criminals new ways to deceive people, impersonate trusted individuals and exploit personal information.

The challenge is not simply that online threats are becoming more sophisticated - it is that many people believe they are better equipped to recognise these threats than they may actually be.

Avast’s 2026 Safe Tech Report, released by Gen Digital, highlights this growing concern. The research found that 56% of adults worldwide believe they could identify convincing AI-generated or manipulated content, while 52% have already shared, or considered sharing, sensitive information with an AI tool.

These findings reveal what Avast describes as an “AI confidence trap”: the assumption that being digitally aware automatically means being digitally safe.

Deepfakes and AI-generated content are making it increasingly difficult to distinguish genuine interactions from carefully constructed deception. Fraudsters can use manipulated videos, cloned voices and realistic messages to impersonate colleagues, family members, executives or well-known public figures. 

For businesses, this creates opportunities for more convincing phishing attacks, fraudulent payment requests and impersonation scams. An employee who receives what appears to be a voice message from a manager requesting an urgent payment may be more likely to comply if the voice sounds familiar and the circumstances seem credible.

The danger is that people often rely on visual or audio clues to determine whether something is authentic. As AI-generated content becomes more convincing, these instinctive checks become less reliable.

Gen Threat Labs reported that, among tens of millions of video encounters involving AI worldwide during the first half of 2026, approximately one in 30 was flagged as a scam. While this represents a fraction of the total, it demonstrates how AI-enabled deception can operate at scale.

Cyber criminals are also continuing to exploit familiar psychological triggers: urgency, fear, trust and offers that appear too good to be true. The technology may be evolving, but the tactics behind many successful scams remain remarkably consistent.

The report highlights another concern: how readily people share personal and professional information with AI chatbots.

One in five adults surveyed had shared, or considered sharing, a CV with an AI chatbot. The same proportion had shared, or considered sharing, a work document.

AI tools can be useful for summarising information, drafting communications and improving productivity. However, users do not always consider what happens to the information they submit.

Depending on the platform and its privacy settings, information entered into an AI tool may be stored, processed or used in ways users do not expect. Sensitive information could include confidential business documents, customer records, financial statements, identity documents and private correspondence.

For organisations, this introduces an additional dimension to cyber security. Employees may unintentionally expose confidential information while using AI tools for legitimate work purposes, particularly when clear guidelines on acceptable use are absent.

Businesses should establish practical AI-use policies that explain which tools are approved, what information may be shared and how sensitive data must be handled. Employees should also understand that removing a document from their own device does not necessarily remove information already submitted to an external service.

The principle is straightforward: if information would be inappropriate to disclose publicly, it should not be uploaded to an AI tool without first confirming that the platform is approved for that purpose and provides suitable data protections.

Interestingly, the Avast report suggests that many people already recognise common scam indicators but do not always act on them.

Among the warning signs respondents were most likely to dismiss were offers that seemed too good to be true (42%), requests for personal information to verify an identity (39%) and pressure to act quickly (37%).

This highlights an important distinction between knowing what a scam looks like and making safe decisions when confronted with one.

A message may appear legitimate, an offer may seem attractive, or a request may arrive at a particularly busy moment. Under pressure, people can overlook inconsistencies or act before verifying the source.

For businesses, awareness training must therefore go beyond teaching employees to identify suspicious emails. It should encourage a culture of verification, where staff feel comfortable questioning unusual requests, confirming payment changes through trusted channels and reporting suspicious activity without fear of blame.

Individuals should adopt the same approach. Unexpected requests for money or sensitive information should always be independently verified, even when they appear to come from someone familiar.

The findings reinforce the importance of combining cyber security technology with informed user behaviour. No single precaution can eliminate every threat, but a few consistent habits can reduce exposure.

Share less by default. Avoid submitting sensitive personal, financial or business information to AI tools unless necessary and authorised. Review privacy settings and understand how a service handles submitted data.

Pause before responding. Urgent requests, unexpected links and offers that promise extraordinary returns deserve additional scrutiny. Contact the person or organisation through a trusted, independently verified channel before taking action.

Use layered digital protection. Keep devices, operating systems and applications updated, use strong and unique passwords, enable multifactor authentication where available, and use reputable security software to help detect malicious websites, suspicious links and other online threats.

Speak up and share knowledge. The report found that 41% of adults delete suspicious messages or activity without telling anyone. Reporting potential scams and discussing new tactics with colleagues, friends and family can help prevent others from becoming victims.

AI is not inherently a threat, and its benefits should not be overlooked. The challenge is learning to use it responsibly while recognising that increasingly realistic digital content can be manipulated and that information shared online may carry risks beyond the immediate interaction.

Avast’s 2026 Safe Tech Report serves as a reminder that cyber security is not simply about identifying what looks suspicious. It is about developing habits that remain effective even when deception is convincing.

For businesses and individuals alike, the goal should be to approach new technology with curiosity, but not blind trust.

As AI continues to reshape the digital landscape, the strongest defence is a combination of informed decision-making, responsible data sharing and reliable cyber security protection.

Avert IT distribution: Your partner for Avast in Africa

For businesses and resellers across Africa, Avert IT Distribution provides access to Avast cybersecurity solutions, backed by local licensing support, technical enablement, training and dedicated partner assistance.

With nearly two decades of experience in IT and cybersecurity distribution, Avert ITD combines global technology with regional expertise to help resellers deliver trusted protection to their customers. Whether you serve SMBs, enterprises, public sector organisations or home users, our team can help you expand your portfolio with proven security solutions and the commercial support needed to grow your business.

Resellers interested in offering Avast products can contact Avert IT Distribution to explore partnership opportunities, licensing options and support tailored to their markets.