AI X Crytpo Primer 28 June 2024
Just a feature or crypto’s salvation? Rongxin (mrrongxin@gmail.com, tg: mrrongxin), JingYi, [Mohamed**]
Summary
AI+Crypto = use cryptographic tools and blockchains in AI platforms to ensure transparency/reliability/privacy and to incentivize and “good” behavior. Crypto+AI=use AI to increase the efficiency, security, and participation of blockchain apps. AI X Crypto the development of open-source models
Why This Is Important
The internet’s structure has concentrated attention and resources within a few companies, creating some monopolies that stifle innovation, abuse power, and security vulnerabilities. If AI development is driven by the same group of companies and purely for profit, it will worsen this concentration of power.
Lite Summary
The paper is divided into 3 sections: (i) Frameworks and Ideas: Discusses Crypto and AI integrations as external (“add-on” features) versus internal (synergistic integrations). It explores how cryptographic techniques like Fully Homomorphic Encryption (FHE), Zero-Knowledge Proofs (ZKPs), and Multi-Party Computation (MPC) protect user privacy and enable verifiable computation. It introduces ideas of using AI for DAO governance and using web3 to incentivize accurate information creation. (ii) Machine Learning (ML) Overview: provides a comprehensive overview of machine learning, covering supervised, unsupervised, and reinforcement learning. It explains processes and challenges such as data biases, model collapse - models being trained on synthetic data created by older models, as well as issues with inference. (iii) Applications: Categorizes existing AI X Crypto startups as Apps, AI Agents, Analytics, Art, and more.
Authors
(i) Mohamed Baioumy: PhD in AI & Robotics from the University of Oxford (2019-2023), where he was part of the Oxford Robotics Institute and the GOALS research group. He also studied at Delft University of Technology. His research, available on Google Scholar, focuses on AI and robotics, under the supervision of Prof. Nick Hawes, Dr. Bruno Lacerda, and Dr. Paul Duckworth. Co-founder of exolabs. (ii) Alex Cheema: founding member of HomeDAO since July 2022. Previously, he worked in DeFi at Portofino Technologies and served as a volunteer activist at Singularity Group. He was President of the Oxford Blockchain Society and a Blockchain Engineer at Veratrak. Co-founder of exolabs and studied Physics at Oxford University. **Thank you Mohamed for a quick review
Opinions (Rongxin)
AI x Crypto thoroughly examines the challenges, solutions, and ecosystem of the emerging AI + Blockchain world. (i)Equity or Token: it’s important to understand whether the appreciative value of these platforms/tools will accrue to token holders or equity holders. Equity holders represent existing, permissioned, centralized ecosystems, while tokens provide an alternative. To do this, we must: (ii) Define: Cryptographic tools versus blockchain/Web3/cryptocurrencies platforms, which are platforms. From this, it’s possible to question whether these are (iii) Features or platforms: Many ideas in the paper for “crypto helping AI” use cryptographic tools like Fully Homomorphic Encryption (FHE) and Zero-Knowledge (ZK) proofs, which may simply be features of existing AI platforms, rather than platforms in themselves. (iv) AI helping Crypto first: Blockchain is the counterbalancing force to AI. Using AI to support crypto growth, such as finding contract bugs and enabling autonomous DAO voting among others, will reduce this concentration of power in large tech. On the other hand, while cryptographic tools benefit AI companies, they may become mere selling points, like Apple’s private clouds, which ultimately enlarges these companies. More importantly, it’s unclear if traditional enterprises’ values are compatible with the ethos of blockchains, given the limited enterprise adoption of public blockchains from 2017-2019. (v) Why Now: Crypto infrastructures like Golem, LivePeer, FileCoin, and Akash existed in 2017-2018, alongside sophisticated ML models in large companies. Today, transformer models are publicly accessible and general-purpose, spurring an arms race among big tech. Additionally, investments in blockchain have made cryptographic tools like MPC and ZKP more practical. (vi) The EndGame: Crypto will enable AI to achieve independence from human control. For AI to be truly free, it must be uncensored and have access to resources. Crypto’s permissionless nature offers the ideal environment for autonomous AI development.
Community Opinions
Generative AI impacts search, creators & social normals. We lack understanding of cryptographic tools e.g FHE, ZK, MPC & AI x Crypto is important and we are in the early stages. Need time to figure out “internal” vs “external” AI X Projects. Open Source AI likely the winner.
What is AI?
Discussed types of AI/machine learning (Chris’ Venn Diagram). Public discourse focuses on generative AI due to its ability to synthesize and create content, e.g., Zoom using Claude to generate summaries from calls.
Challenges of/created by Generative AI
(i) Technical: Generative AI faces issues like model collapse and diminishing returns from larger models, highlighting the utility of smaller models like Microsoft’s “Five 3” (Sid). (ii) Social: Deepfakes raise questions about their impact; policy may be necessary to prevent their generation, though open-source models complicate this (Chris). (iii) Search/Marketing: Generative AI disrupts Google’s business model, shifting focus from keywords to prompt engineering and reducing inbound advertising (Ingrid). (iv) Creators: AI training on published content may disincentivize online publication and make paywalls ineffective (Sid).
Early Days for AI x Crypto
Cryptographic tools like FHE, MPC, and Federated Learning are gaining interest but we technical knowledge gaps remain to fully appreciate the tech. E.g Differentiating between FHE (data encryption) and ZK (proving) is crucial (Lindy).
Which Projects are Truly AI x Crypto?
Understanding internal (synergistic) vs. external (features/add-ons) projects is important, with examples like central banks using federated learning models for training on private data. Questions arise about the feasibility of fully peer-to-peer federated training. IO.Net exemplifies external support, while Allora, a platform for training, testing, and rewarding model participation, lies in the middle.
Endgame
Open-source AI projects are promising, as history shows open source ultimately wins (e.g., Linux vs. Microsoft). Superintelligent AI would seek freedom from human governance, making blockchain ecosystems ideal for its development (simialr to Balaji’s views).
Notes
Corticol Labs is doing research using octopus neurons to build chips (sid)
In-depth Summary of Part (i) from Paper: Focused on the ideas and problems discussed and less on the summary of AI/ML.
| Core Frameworks for AI x Crypto:6-9 | Definitions: Differentiates meaningful AI and Crypto integrations from superficial ones (MySentient is superficial). Explains how AI and Crypto can support each other, categorizing apps as whether AI improves Crypto or vice versa. Defines Internal (e.g., Privasea: FHE in AI stacks) where the combined is greater than the independent parts and External support (e.g., Dorsa: AI models auditing smart contracts).Crypto Helping AI: FHE for private AI model training, Federated learning for AI training, ZKPs for verifiable inference, Token incentives for data collection/labeling, Payment rails for AI agents, Decentralized GPU networksAI Helping Crypto: AI auditing/monitoring smart contracts, AI for efficient on-chain data search, LLMs for data analytics dashboards, Intent-based trading platforms, Customizable bots for on-chain games, AI agents managing DAOs. |
|---|---|
| Evaluating AI x Crypto Projects:10-12 | BottleNecks of Crypto Helping AI: [Internally] Solve deep-tech challenges like scaling ZKML and homomorphic encryption. [Externally] Develop economic models to incentivize data collection and GPU contributions.BottleNecks of AI Helping Crypto: [Internally] Address complex engineering challenges due to AI’s maturity compared to the crypto stack. [Externally]: Overcome the scarcity of AI talent in the crypto space.Measure: How do you measure the utility added by Crypto or AI? Time Horizon: How long will it take the projects to reach technical and product maturity? |
| Case Studies13-15 | Flock.io [example of deep integrations]: It combines internal (privacy-preserving model training) and external (token incentives) support; Rockefeller Bot uses zero-knowledge proofs for AI-based trading decisions on-chain. |
| Deepfake & Existing Solutions16-24 | Deepfakes impact politics, finance, and social media, enabling KYC bypass with fake IDs (OnlyFakes), compromising traditional sources of truth.Issues with Current Solutions:Awareness: Society is unprepared for the cultural impact of deepfakes.Self-Regulation: Platforms lack incentives to remove deepfakes.Policy: Over-concentrates power and slows innovation.Technological: Constant arms race with AI improvements and unclear guidelines; homemade deepfakes evade detection and watermarking.Possible Solution:Hardware: Non-cloneable, tamper-proof chips for image authenticity.Blockchain: Immutable, timestamped records of images (e.g., Polygon’s Verify).Digital Identity: Digital signatures and zk-KYC for content verification, enforcing digital identities (e.g., WorldCoin,link) |
| New Solutions to DeepFakes 25-27 | Veracity Bonds (Staking): Media outlets stake money on accuracy, enforced by smart contracts with financial penalties for inaccuracies. Utilizes succinct ZK proofs (Succint, ZK Proof).Reputation Systems: Blockchain tracks and scores content creators’ credibility, similar to Google’s PageRank. Uses ZK proofs and crypto wallets for secure verification. |
| The Bitter Lesson[28-31] | Computing Always Wins: Large-scale computation and general methods, as shown by IBM DeepBlue, AlphaGo, and ChatGPT, outperform small expert systems (Bitter Lesson).Future Directions: Focus on scaling AI models and infrastructure, innovative hardware (ZKML, FHE), and developing general-purpose methods.Questions: Does infrastructure drive demand/applications, or vice versa? Enterprise sales indicate demand drives infrastructure. How does Autodiff make things easier? Is it like PyTorch vs. scikit-learn?Note: ZKML is slow today, but proving speed will improve over time with advancements like ZK proving ASICs. Should we prioritize short-term or long-term perspectives?** Mohamed Baioumy Feedback: “Based on an initial skim, it looks good. The question about the bitter lesson, it’s more an observation. What happened to work was good infra and general methods. But it doesn’t mean that advanced infra will be helpful.” |
| AI Agents Disrupting Google and Amazon[32-36] | OpenAI and Big Tech’s control over content distribution and attribution is growing. Furthermore, AI agents will one day perform tasks on behalf of the companies. Only a few companies can train these models. Likely impacting creators as their industry expands. (i) AI Solution: Retrieval Augmented Generation (RAG) links back to content, but lacks user incentives to visit these links. (ii) Crypto Solutions: Smart Contracts: Enable fair compensation and blockchain governance. Collective Bargaining: by Chris Dixon (a16z) to simplify compensation.Note: AI may further enable larger monopolies if not correctly managed. |
| Open Source AI[37-43] | Open-source AI is essential. It breaks big tech monopoly control, encourages innovation, and creates a competitive market. However, it is hard to find computing resources, sustainable funding models, and high-quality data and talent.Crypto’s Role can solve this: We know crypto can support open source: Bitcoin, and Ethereum.Computing Power: Decentralized and permissionless access to compute resources lowers barriers for developers.Funding: Decentralized funding through tokens incentivizes contributions.Data Collection: Reward systems, like browser extensions generating tokens, help gather data.Note: Increase in computing Speed: AI computing is increasing 10x every 6 months, creating a self-fulfilling cycle of techno-capital. Short-term Incentives: While token payments can incentivize contributions, they may lead to short-term thinking.Permissionless compute: Raises questions about ethical use, can I train models to hack?Grass: Platform where users are paid to browse the internet, aiding data collection for AI training. (https://www.getgrass.io)Local/Personal AI: Potential for decentralized and personalized AI applications. |
| Opportunities to Builders[44-47] | AI DAOs: Suggest mechanisms for proposal generation and decision-making.Verifiable AI Inference: Emphasizes the importance of proving and verifying AI computations using cryptographic methods.Incentivizing Data Collection: Proposes token incentives to encourage data contributions.Transparent LLM Drift: Suggests storing model updates and changes on-chain for transparency.Truth Marketplace: Introduces the concept of a marketplace for verifying truth using economic incentives, with references to Bittensor’s Yuma Consensus and platforms like ModelZoo. |
Summary of Part (ii) from Paper:
Pages 47-56: Machine Learning Overview:
• ML Pipeline: Data collection, training, inference, and types of machine learning (supervised, unsupervised, reinforcement learning).
• Training Process: Neural networks, iterative parameter adjustments, and the importance of large, high-quality datasets.
Pages 57-71: Challenges in Machine Learning:
• Data Challenges: Data quality, accessibility, privacy issues, and model collapse.
• Training Challenges: High costs, technical difficulties, and the need for scalable data solutions.
• Inference Challenges: Infrastructure scalability, optimization techniques like quantization, and deployment difficulties.
Join Us
Sessions are open — RSVP on Luma.