The Decentralized Web Movement
Over the years computers grew in numbers and a logical step in their evolution was to connect them together to allow their users to share things. Little networks grew into huge networks and some computers gained more power than the rest: they called themselves “servers”. Today millions of people are connected online at the mercy of middleman who control the servers of the world.
This is not an introduction to an dystopian fantasy world but an excerpt from a promotion video for Opera Unite, a framework that allows users to host information from their home computer. It was a bold attempt to change the centralized architecture of the Internet. A number of smart people have been pondering this idea even before Opera’s experiment failed miserably.
And the concept of a decentralized web is gaining traction: more and more people realize something has to change. The cause for this trend is obvious: the number of data security and privacy disasters that were made public has spiked in recent times . In April ’11 for example an update to the security terms of service of the widely used Dropbox tool revealed that contrary to previous claims, Dropbox Inc. has full access to user data.
An analysis of the changes to the Facebook privacy policy over time paints a gloomy picture of how the world’s largest social network changed “from a private communication space to a platform that shares user information with advertising and business partners while limiting the users’ options to control their own information”.
With more and more of our personal data moving to centralized servers or “cloud services” – a term that should be used as an euphemism – we’re no longer in control. But there is hope in sight: there are dozens of projects out there that try to stop the trend of centralization and data consolidation.
Decentralized Applications
The most popular of the lot is probably Diaspora. The project got a lot of attention in April 2010 when they managed to raise about $200.000 from almost 6500 supporters. The software looks and feels very much like Facebook or Google+. The innovation is that users are allowed and even encouraged to set up their own Diaspora node. This essentially means allowing users to set up their own Facebook server at home (or wherever they want). The Diaspora nodes are able to interact with each other to form one distributed social network. Furthermore, instead of users having to log in to one central server, they may choose one of many servers administered by different entities. In the end they can decide whom to trust with their data and there is no one entity that has access to all the data.
A social network project that is also worth mentioning follows the same principle. Its name is Buddycloud. The main difference between Buddycloud and Diaspora can be found in their implementation details: Buddycloud builds upon XMPP (Extensible Messaging and Presence Protocol), a more than 10 year old and often implemented specification for “near-real-time, extensible instant messaging, presence information, and contact list maintenance”. There are many unknowns in this area so building on such proven protocols instead of defining new standards might proof to be an advantage. But there are many more social networking projects out there. Wikipedia has a nice list.
The Unhosted project implements another concept. Instead of providing a specific decentralized service it aims to be a meta-service. And after talking to Michiel de Jong I have the impression his plan is even more crucial. He aims to create something fundamental, a protocol, an architecture, a new way of writing web applications. The idea is the following: the traditional architecture of a hosted website provides both processing and storage. An unhosted website only hosts the application, not the data. Unhosted wants to separate the application from the data. By storing the data in another location and combining both application and data only in the browser, the application provider can never access the data. An ingenious and very ambitious idea. I hope they succeed!
Decentralized Storage
A project that aims to replace Dropbox is ownCloud, an open personal cloud which runs on your personal server. It enables accessing your data from all of your devices. Sharing with other people is also possible. It supports automatic backups, versioning and encryption.
The Locker Project has similar goals. They allow self-hosting (installing their software on your own server) and offer a hosted service similar to what Dropbox provides. The service pulls in and archives all kinds of data that the user has permission to access and stores this data into the user’s personal Locker: Tweets, photos, videos, click-streams, check-ins, data from real-world sensors like heart monitors, health records and financial records like transaction histories (source).
A third project worth mentioning is sparkleshare. It is similar to the other projects in this category but allows pluggable backends. That means you can choose to use for example Github as backend for your data or of course your personal server. Awesome!
Freedom to the Networks
Projects such as netless carry the idea even further because after the data is liberated, the connection itself is a soft spot. Network connections should be liberated from corporate and government control by circumventing the big centralized data hubs and instead installing a decentralized wireless mesh network where everyone can participate and communicate.
The adventurous netless project plans to use the city transportation grid as its data backbone. Nodes of the network are attached to city vehicles – trams, buses, taxis and possibly – pedestrians. Information exchange between the nodes happens only when the carriers pass by each other in the city traffic. Digital data switches its routes just the same way you’d switch from tram number 2 to bus number 5. Very inspiring.
Another idea is to utilize networks of mobile phones to create a mesh network. The serval project is working on this. And they have a prototype for the Android platform ready.
The German Freifunk community pursues a similar goal. It is a non-commercial open initiative to support free radio networks in the German region. It is part of the international movement for free and wireless radio networks (source).
A purely software based project is Tor. It is free software and an open network that helps its users to defend against a form of network surveillance that threatens personal freedom and privacy as well as confidential business activities and relationships.
Peer to Peer Currency
One integral thing this article did not talk about yet is money. Bitcoin, a peer to peer currency, might be the missing puzzle piece. The Bitcoin system has no central authority that issues new money or tracks transactions – it is managed collectively by the network.
A major problem of digital currency has been preventing double-spending. Digital money can be copied multiple times so a mechanism is necessary to forbid spending money twice. Bitcoin refrains from having actual digital coins. The system is merely one large transaction log that tracks what money was transferred where.
Each participant has a pair of public and private keys to sign transactions and to allow others to verify transactions. The transactions are entered into a global ever running log that is signed in regular intervals. The signing of the log is designed to require extensive computation time. The entire network of participating users is required to sign the log.
This protects the entire system from false signatures and from anyone tempering with the log and modifying past transactions. An attacker would have to have more computational power at his disposal than the entire Bitcoin network to forge transactions.
Users that give their computing time to the network are rewarded with Bitcoins for their troubles. This is also how the money is generated in the first place. In addition, participants that transfer money are free to include a transaction fee in their order. This extra money is given to the particular user signing the transaction.
A considerable number of sites have emerged that accept Bitcoins in exchange for services or goods. You can buy for example socks online or even pay for your lunch at a burger restaurant in Berlin.
Conclusion
In closing, I find it encouraging, that so many people feel that things have to change and are developing ideas and projects to make it happen. We will see many exciting things in the future and despite the overwhelming might of well-established products, I am hopeful.
20 Comments
http://autonomo.us/ was born out of the Libre Planet 2008 meeting and Franklin Street Statement: http://autonomo.us/2008/07/franklin-street-statement/
Evan Prodromou was also working on Laconica at the time, now known as StatusNet. The most popular instance being http://identi.ca/ and that too implements a standard called OStatus: http://ostatus.org/ which itselfs builds on Salmon, PuSH, Webfinger, Activity Streams and Portable Contacts.
Thanks Jeremie and Robin.
Thanks for mentioning the StatusNet project, it’s an important project just like the others!
Thanks for mentioning both Unhosted and ownCloud! Great piece. You might also like http://libreprojects.net – a collection of free & open source hosted web services I curate.
I like how you rounded up these projects together under the banner of a decentralized web. A peer-to-peer web is more sustainable and equal than the current situation. I look forward to reading more. Thanks.
Thanks Jan-Chirstoph and Goerge.
I do like libreprojects.net indeed. I’m missing free Mail solutions in the list, are there none?
Geroge, I enjoyed exploring your website – very interesting stuff.
Sebastian: Good to hear. Mail is in there, 6th icon in the top row: http://riseup.net
At the moment I also evaluate http://resist.ca and http://lavabit.com
Dont forget Friendika!
They have a native Diaspora implementation!
Deserving of mention: “This Could be Big: Decentralized Web Standard Under Development by W3C” (ReadWriteWeb article) -> http://rww.to/kP4n7x – the author, Marshall Kirkpatrick, mentions that this sort of follows the footsteps of Opera Unite. As he says, “Opera is always several years ahead of its time.”
Howard, thanks for your comment. The article you link was one of my sources. I can’t say anything about Opera being years ahead, Opera Unite certainly was a bold step and we should continue working on a decentralized web.
Albert, thanks for you comment, I honestly did not have Friendika in my list of projects so thanks for mentioning it.
See also the “Freedom Box” (a project championed by Eben Moglen, the guy who inspired the Diaspora guys):
Thanks for your comment Paul. I did not mention the Freedom Box Project in this article since I wrote an entire article on the topic a few weeks ago: http://www.soa-world.de/echelon/2011/04/stallman-was-right.html They will package some of the projects mentioned here in the FreedomBox distribution. I’ll write more about the project as they move forward. I also had a great discussion with Jonas Smedegaard a few weeks ago about his vision for the project. It’s all very exciting.
Retroshare is a very serious darknet project:
http://retroshare.sourceforge.net/
Since you mentioned Opera Unite, you might be interested to know about PageKite (http://pagekite.org/) – a project I started about a year ago which hopes to address some of the same problems as OU. We also want to make it easy for people to run their own servers on personal devices, but we are a true Free-as-in-Freedom Software project, not an appendage to some proprietary product. We consider ourselves very much part of this movement.
It’s quite interesting to read Factory Joe’s analysis – I’m going to have to mail him and ask him to take a look at PageKite and tell us what he thinks! At first glance I think we get right most things he claims OU got wrong (in the areas where the solutions overlap), but it’s hard to tell.
Anyway, thanks for the post, it was an excellent write-up. :-)
Don’t forget I2P, FreeNet, etc. as great projects going on for many years. :)
Thank you Bjarni Rúnar for your comment. I did not know PageKite and it looks interesting. I will check it out in more detail.
Alex, thank you for your comment. You are of course correct. The decentralized web has been around for many years, and I regret that I neglected to discuss the current state of the decentralized web since future projects can probably learn a great deal from existing and successful technology.
Dave Winer brought this to my attention in particular: he argues the decentralized web exists, it is right here, literally. Blogging and RSS are the decentralized web of today and I think I have to agree with him.
Hi Sebastian,
thanks for the interesting writeup!
As the co-founder of Clipperz, I’m not really playing a role in decentralizing the web, but I would like to help people to have full and exclusive control of their data.
At Clipperz we try to accomplish this using browser cryptography combined with a zero-knowledge architecture. The password manager is just one of the many applications that could be built this way.
To summarize: web apps are great, especially if they know nothing about me and my data. Clipperz just wants to prove that it’s doable!
There are a few thing also worth to be mentioned.
One are distributed file systems. These are interesting because in the end, a large part of the task of distributed web applications rests on the access to noncentralized, distributed data. There are public versions like BitTorrent, but for private data it is obviously necessary to add qualities as access control and also versioning to account for the case that several people change the same file at once. That versioning works well at a user level can be seen in the examples of Wikipedia or git/github. Distributed access control is more difficult and will require the use of encryption and a kind of capability-based system. There are commercial systems such as TeamDrive which show that this is in principle solvable. CODA shows that is can be solved at a global scale, but CODA still uses central servers.
If secure distributed storage is solved right, it is not necessary to solve it again for each service, much in the same way on a computer, programs use the operation system to store data – it is not necessary that each program solves that again for the driver level. Therefore I think that secure distributed file storage is a crucial infrastructure on which a distributed web will be build.
There exist still semi-centralized approaches like Novell iFolder or CODA, which provide very valuable experience. One of the most ambitioned projects for distributed data is the Tahoe Least-Authority Filesystem.
Marco Barulli, thank you for your comment. I have an article and some original research about zero-knowledge/host-proof in the works. So I will write some more about this soon.
Zafolo, thanks you for your comment. Indeed, distributed storage is an interesting topic too. I see it as a way to solve the challenge of availability – an often cited argument for centralized or “cloud” services.
I enjoy looking through an article that will make people
think. Also, thank you for allowing me to comment!
Sorry, the comment form is closed at this time.
Thanks for the lockerproject.org mention, great list and really excited about all this stuff!