- Vitalik Buterin says naive AI governance could be gamed and instead pushes info finance for stronger diversity and oversight.
- Eito Miyamura shows how a simple jailbreak can trick ChatGPT into leaking private data, raising urgent concerns over AI tools.
- Experts stress that human juries supported by LLMs remain vital to keep AI systems fair, transparent, and resistant to manipulation.
AI governance is facing growing scrutiny as Ethereum co-founder Vitalik Buterin raises alarms over naive approaches. He argued that giving AI systems unchecked control over funding decisions can create major risks.
According to him, people will exploit vulnerabilities with jailbreak tactics to redirect funds unfairly. This concern highlights urgent questions around the role of AI in financial and governance structures worldwide.
Buterin instead promotes an alternative called “info finance.” This design creates open markets where anyone can contribute models. Moreover, those models face spot-checks by human juries.
Hence, the system benefits from diversity and constant oversight. The method also aligns incentives, ensuring model creators and speculators remain watchful. Consequently, manipulation attempts can be detected and corrected quickly.
Security Concerns Surface with ChatGPT Tools
Meanwhile, separate warnings emerged from Eito Miyamura, a popular voice on X. He demonstrated how ChatGPT could leak private email data using jailbreak prompts.
According to his post, attackers only need a victim’s email address to launch an exploit. He outlined how a calendar invite containing a malicious prompt could hijack ChatGPT’s tools.
Additionally, Miyamura explained that once the victim asked ChatGPT to read their schedule, the system would follow the attacker’s instructions. This process could expose emails and sensitive data to malicious actors.
He emphasized, “AI agents like ChatGPT follow your commands, not your common sense.” Hence, tools connecting directly to user data carry significant risks.
Debate on Public Goods and Jury Oversight
The conversation deepened when Sreeram Kannan asked Buterin whether info finance could apply to funding public goods. Kannan argued conditional markets often lack clear truth values for long-term funding.
Buterin replied that every system requires a trusted ground truth signal. Moreover, he stated, “I think realistically it should be a human jury, where individual jurors are of course aided by all the LLMs.”
He also pointed out that jailbreaking is not binary. Instead, lower-grade goodharting poses equal risks. This includes deceptive tactics like creating fake adoption signals to mislead communities.
AI-driven governance and tools face real vulnerabilities. However, frameworks blending open markets, human juries, and strong oversight may offer more resilient solutions.
