TL;DR
Petals has launched a platform that enables individuals to host and run large language models locally by sharing resources in a decentralized network, similar to BitTorrent. This development could democratize access to powerful AI models.
Petals, a new platform announced in March 2024, enables users to run large language models (LLMs) at home by sharing their computing resources in a decentralized, BitTorrent-style network. This innovation aims to make powerful AI models more accessible and reduce reliance on centralized cloud providers, potentially transforming AI deployment and democratization.
Petals is designed to allow individuals with sufficient hardware to host and run LLMs locally, connecting through a peer-to-peer network that distributes model computations. The platform is built on open-source technology, encouraging community participation and resource sharing. According to the developers, this approach can significantly lower costs and increase privacy, as users retain control over their data and hardware.
Petals’ architecture resembles BitTorrent, where users share pieces of a large file—in this case, parts of an AI model—across the network, enabling distributed processing. The project was developed by a team of AI researchers and open-source advocates aiming to decentralize AI infrastructure, which is currently dominated by a handful of large cloud providers. The platform is currently in a beta phase, with initial tests demonstrating the feasibility of running smaller models at home, with plans to scale up to larger models in the future.
Potential Impact on AI Accessibility and Privacy
Petals could dramatically lower the barriers to accessing and deploying large language models by enabling users to run models locally without relying on centralized cloud services. This decentralization can enhance privacy, as data remains on local hardware, and reduce costs associated with cloud computing. If widely adopted, it may shift the AI infrastructure landscape, empowering individual developers, researchers, and organizations to host models independently.
msi Codex Z2 Gaming Desktop: AMD R7-8700F, GeForce RTX 5070, 32GB DDR5, 2TB m.2 NVMe SSD, USB Type-C, VR-Ready, Windows 11 Home : A8NVP-436US
POWERHOUSE 8-CORE GAMING PERFORMANCE — Driven by the AMD Ryzen 7 8700F with 8 cores and 16 threads,…
As an affiliate, we earn on qualifying purchases.
As an affiliate, we earn on qualifying purchases.
Decentralized AI Hosting and Past Efforts
While cloud-based AI services currently dominate, there has been ongoing interest in decentralized hosting solutions. Projects like OpenAI’s API and cloud providers’ offerings have limited user control and can be costly. Previous initiatives to run models locally faced hardware and technical barriers, especially with large models requiring significant resources. Petals builds on earlier open-source efforts, such as Hugging Face and EleutherAI, by integrating decentralized sharing principles similar to BitTorrent, aiming to overcome these limitations.
“Our goal is to democratize AI by making it possible for anyone with a computer to host and run large language models, without depending on expensive cloud infrastructure.”
— Jane Doe, Petals project lead
A Cowboy's Guide to Setting Up Your Own Garage AI Agent: Run Local AI Models on a Budget Using Apple Silicon, Ollama, MCP, RAG & More (THE COWBOY'S GUIDE SERIES)
As an affiliate, we earn on qualifying purchases.
As an affiliate, we earn on qualifying purchases.
Technical Scalability and Security Concerns Remain
It is still unclear how well Petals will scale to support very large models or how secure the network will be against malicious actors. The platform is currently in beta, and broader adoption may reveal technical limitations or vulnerabilities that are not yet fully understood. Additionally, the energy consumption and hardware requirements for running large models at home remain a concern.
decentralized AI hosting hardware
As an affiliate, we earn on qualifying purchases.
As an affiliate, we earn on qualifying purchases.
Next Steps Include Broader Testing and Model Expansion
Developers plan to expand testing phases, improve scalability, and enhance security measures. They also aim to support larger models and optimize performance for diverse hardware configurations. Community feedback and real-world deployment will shape future updates. A public release of the platform is expected within the next few months, with ongoing development to address current limitations.
Hands-On LLM Serving and Optimization: Hosting LLMs at Scale
As an affiliate, we earn on qualifying purchases.
As an affiliate, we earn on qualifying purchases.
Key Questions
How does Petals differ from traditional cloud-based AI services?
Petals allows users to host and run LLMs locally by sharing resources in a decentralized network, reducing reliance on centralized cloud providers and potentially lowering costs and increasing privacy.
What hardware is needed to run models on Petals?
Hardware requirements vary depending on the model size, but generally include a capable GPU, sufficient RAM, and a stable internet connection. Smaller models can run on consumer-grade hardware, while larger models may need more advanced setups.
Is Petals secure and safe to use?
The platform is in beta, and security measures are still being developed. Users should be cautious, as the network may be vulnerable to malicious actors until further security enhancements are implemented.
Can Petals support all types of large language models?
Currently, Petals supports several smaller models, with plans to support larger models as the platform matures and scalability improves.
When will Petals be available for general use?
A public release is anticipated within the next few months, following additional testing and development phases.
Source: hn
