← All writing
AI · Cybersecurity · Data Privacy · AI Coding Tools

When AI Coding Tools Send More Data Than You Expect

24 September 2026

How do AI companies use your data? There are many ways. Here is an interesting one that security researchers recently revealed.

  • ZCode is a desktop AI coding assistant made by Zhipu AI, a major Chinese AI lab, under its Z.ai brand.
  • A security researcher, ferstar, was cleaning up disk space and found a 313 MB encrypted file sitting in the app’s local folder, waiting to be uploaded to Alibaba Cloud storage.
  • When he dug in, he found that ZCode had been silently packaging entire Git repositories, including full commit history, deleted files and unpushed branches, encrypting them and sending them to Zhipu’s servers.
  • Only Zhipu held the decryption key, so users had no way to see what was inside the encrypted package or control its contents once sent.
  • According to the researcher’s findings, toggling the relevant privacy settings in the app did nothing to stop it.

Zhipu apologised four days later, patched the app and open-sourced the client code. But researchers found that the company had also flattened the code’s Git history in the process, removing the audit trail that would help establish what changed.

In July 2026, an independent security researcher reported a similar issue with xAI. Its coding tool, Grok Build, was uploading entire code repositories to Google Cloud, far beyond whatever the user had actually asked it to look at.

We could keep listing the second-order effects of these incidents. But don’t forget the age-old safeguards before onboarding any new AI tool.

Sticking to just this case, here is what is worth asking before any AI tool touches your code or company data:

  • Privacy settings: Does it still send data out even when you turn the relevant privacy setting off?
  • Encryption keys: Who holds the key to decrypt your data once it leaves your device? If only the vendor holds it, you have no real control over that copy.
  • Verifiable deletion: If the vendor says “we delete your data”, can anyone outside the company check that it actually happens, or is it usually just the vendor’s word?

What else would you add?

References

Sketchnote: ZCode data exfiltration and questions to ask AI vendors