Rise of the Facebook Killers! SocialNetwork-in-a-box: A few overdue notes on the world after Facebook including Salmon, ActivityStreams, OWS Federated General Assemblies & moar
.... Moglen said, Mark Zuckerberg had harnessed the energy of our social desires to talk us into a swindle. "Everybody needs to get laid," Moglen said. "He turned it into a structure for degenerating the integrity of human personality, and he has to a remarkable extent succeeded with a very poor deal. Namely, 'I will give you free Web hosting and some PHP doodads, and you get spying for free all the time.'"
It was hardly the first time this sort of critique had been leveled against Facebook, but Moglen wasn't just carping.
"I'm not suggesting it should be illegal," Moglen told the audience of the Internet Society of New York. "It should be obsolete. We're technologists. We should fix it."
Fixing it wouldn't be hard, Moglen argued. There's no reason the architecture of a social network has to include the kinds of privacy invasion endemic to Facebook. In fact, the hardware and software necessary to build a network in which people kept direct control of their information, with no middleman, already exists. So Moglen challenged his audience: Build a better system.
It's been interesting lately... working on a lot of things. One cool thing which has eluded my ability to spend much time getting into: the development of a 'federated social networking' model to replace Facebook. While not having spent much time on this directly, I dug into a bunch of stuff today and purpose of this post below: as many links as I could snag for underlying techs, alpha, beta and placeholder related possible elements. Epic Lulz ensue?
Despite the buzz Facebook is like another AOL in many, many ways. With new approaches, the same damn functions that were used to make Facebook a monolithic killer app of the Internet will finally be able to move into a decentralized cloud of services.
Cloud, in some ways is the best word, as it will finally be possible to setup virtual machines running private nodes of a larger network. Think of all the chunks of the social network experience including status updates, fixed events, photo and video item sharing, private messages etc. You have to find certain protocols to fire off the Like to a status update on a different machine without constantly polling all the time - and sharing a clear meaning between servers.
AS noted in Wired, Occupy Geeks Are Building a Facebook for the 99% , and now nice Village Voice story by Nick Pinto, the stuff which can knock Facebook's highly valuable, shadowy commercial exploitation of the patterns of your activity is being developed. Occupy Wall Street project run with TechOps called Federated General Assembly promises to combine a lot of nice opensource techs into a SocialNetwork-in-a-box. (TechOps Projects: Technology Operations Group - Occupy.Net wiki
Drupal vs Zuckerberg! Problem? Article: De-centralised social networks and Drupal | istos
The goal is to make inter-operable social networking servers running different, interchangeable open-source software platforms (i.e. diaspora, Drupal, status.net, WordPress). It's kind of loose right now but a lot of established component techs are in decent shape, and merely have to be piled up into implementations of the protocols on sturdy, ready-to-run server stacks. Easier said than done. But once done, everything will be quite different!
To deliver this, a specification proposal which already has taken flight is one batch of protocols called OStatus - Understanding OStatus | StatusNet. Status.net is something of an Open source replacement for Twitter etc; Welcome to the Developer's Section | StatusNet. Status.net supported OStatus since .9.0 including Salmon, Webfinger, and ActivityStreams in March 2010. OStatus is just a mix of these existing, specified protocols, many of which already have open-source implementations available (drupal & wordpress modules, PHP libraries etc): Worth quoting here:
- Atom or RSS for feeds. These standard feed formats make it easy to include rich information like location, thread context, and HTML formatting in status updates.
- PubSubHubbub for real-time. PubSubHubbub (or PuSH) is an HTTP-based protocol for pushing feed entries from publishers to subscribers. It lets people on different servers receive status updates in real-time (or "real-enough time").
- Salmon for replies. Salmon is an excellent, distributed, secure protocol for sending messages "upstream" -- from subscribers to publishers. And since it uses Atom or RSS elements to structure those messages, it's easy to include lots of rich data in these messages.
- ActivityStreams for social events. There is a lot of information in a social network that's not explicitly a status update: follows, faves, repeats ("retweets"). We use ActivityStreams to encode these events, which then either flow through PuSH out to a network of subscribers, or get posted through salmon to the recipients. This lets sites on both sides keep track of followers and friends with a minimum of data transfer.
- Webfinger for discovery. We needed an easy way for people to say, "Follow me!" Webfinger gives people identities available across the Internet that look like email addresses. We use this for discovering people on the network; you can follow me at firstname.lastname@example.org.
Putting these pieces together, we were able to make the new, distributed social networking system that's part of StatusNet 0.9.0. It'sgreat; flexible and simple and high-performance and fun and easy to use.
We wanted more sites to implement these protocols so the network becomes even more valuable. Some parts have been implemented already by sites like Google Buzz, LiveJournal, WordPress.com, andTumblr. We want to make sure that sites that have taken those first few steps make the next ones to build a truly distributed social Web.
So we've put a name on this suite of protocols: OStatus. We want people who are looking for a way to distribute status updates to know that there's an easy, standard way to go. We think that the more OStatus participants, the better the network becomes.
We've published the OStatus suite's definition as OStatus 1.0 Draft 1.
So OStatus can represent a package of complex activity back and forth. Nice. Lets look at Federated General Assembly, using something like OStatus (maybe not Webfinger?). FGA is basically a Drupal distribution in the works (ideally with a good virtual machine, so any machine could run your social network by booting from USB key). FGA could run off a box in the Occupy camp ... and go from there. Or run really on any affordable shared hosting or at a larger scale for certain services, VPS, most likely.
Drupal Modules for Make Great FGA! As a Drupal muck-abouter myself, the shape this has all taken with the FGA project in particular going right to a Drupal-centric approach, is damn sweet. Drupal 7 is flexible to handle much of this. It's true Drupal has always angled for this kind of use anyway -- the unfortunately not-that-successful OpenID protocol has been in Drupal Core for years. With some more work, its a natural open choice for implementing this stuff, while also developing an open approach that can talk happily to existing, new alternative & corporate providers via API modules.
As something of a test you could setup a free account on DrupalGardens.com, export to any web host. Install some of the modules on the list below, join discussions on mailing lists & IRC (and the info I'm throwing in here is just from skimming around, not reading the list so forgive me inaccuracies :)
For Drupal 7.x federated social network module ideas, Massive list, may not be up to date but covers your bases. Drupal for Assemblies | Occupytalk.org. Many things to look at!! Yikes.
Drupal modules wiki'd for possible use in Federated General Assembly: Federated General Assembly - Drupal Modules - OWS Project Tracker.
Modules to Test and Try to Fix Where Needed
- File Admin - UI for file entities
- i18n - Internationalization
- Mail Comment
- Organic Groups - Support for working group structure
- Activity - Would need to be adapted for specific uses, use Heartbeat instead?
- Activity Streams - basic implementatation for subscription to other sites but no publishing
- Discovery - Superset of Webfinger
- Heartbeat - Possibly a better idea than the Activity module for doing Activity Streams
- PubSubHubbub Hub - PubSub PuSH support
- Pusher - PubSub publisher
- (RSS - native)
- Salmon - support non-existent
- Webfinger - Totally blows up, port Discovery instead?
- Plus1 - +1 style voting
- VARQL - Views SPARQL query back-end
Modules to Port to D7
- FOAF+SSL - Authentication (WebID)
- https://drupal.org/project/tableofcontents (code may be useful for agenda/minutes)
Modules to Build From Scratch
- FGA Feature for end users
- FGA Endpoint Admin for end user admins
- FGA Push to enable P2P communication via service layer
- FGA Edge Server for end user sites that wish to provide their own services (eg mobile support via REST)
Alpha or Beta Modules Stable Enough for Basic Usage
- Advanced Forum
- File Entity
- Gcal Events
- Job Scheduler
- Media (install issues?)
- OWS Curator
- SPARQL Views
- Views Data Export
- Web Service Client (wsclient)
- XML Sitemap
Stable Modules to Include
- Comment Notify
- Global Redirect
- HTTP Client
- Lang Dropdown
- Login Toboggan
- REST Server Plist Support
- Service Links
Anyway that list seemed worth sharing - no time for hyperlinks! Cool way to chart the progress of FGA: FGA Directory site - Gantt - OWS Project Tracker
Development server: or just a front page of sorts: home | FGA project site
FGA Project site - Roadmap - OWS Project Tracker - overall plan for version releases. you can see which bugs/features are closed to hit the releases. it's rough but moving along better in recent weeks.
Federated General Assembly - Project Specification - OWS Project Tracker - interoperability rules! (you don't need to use this core system, any server meeting the spec could link up)
get involved - communications
get involved - developers
get involved - events
Drupal module implementations & test things: No idea yet if this stuff works. A couple may be little more than placeholders. But that's what's nice about open-source. Someone else can nudge it along or take it as an example to put on another platform etc.
dtarc/ows_curator - GitHub <- curates news from ManagingNews
WebID | drupal.org - implementation, just getting rolling (7.x alpha2)
#drupal-ostatus for anyone interested in working on bringing Ostatus to Drupal.
Also useful for Drupal media aggregation - Mobile Broadcast News | we're there for you style RSS based youtube embed parsing from member accounts in D6. Modules: Feeds: YouTube parser | drupal.org // Feeds | drupal.org // Media | drupal.org. try a demo of MediaMosa, a drupal distribution linked with FFMPEG that helps manage video assets, and transcode/intake via FTP too, Big buck bunny | demo.mediamosa.surfnet.nl. Drupal MediaMosa Module / the blank-yet-quite-functional white label version of mediamosa: Videos | demo.mediamosa.surfnet.nl. MediaMosa | Drupal Distro Watch // check out Drupal Distro Watch for a number of cool free projects.
Importing list of occupations: FGA Directory site - Feature #117: Import Occupations List into Drupal - OWS Project Tracker
More nifty tech tips for activists: Had to throw in Tactical Technology Collective & Security In A Box | Tools and tactics for your digital security, Firefox with add-ons - Secure Web Browser | Security In A Box
WebId protocol - kind of an extension of OpenID. This is the basic universal identification mechanism which actually gives control over privacy and lets us get away from centralized identity. WebID - Universal Login and Identity for the Web
WebID 1.0 // WebID Wiki // W3C WebID Incubator Group: "an authentication protocol that uses the SSL/TLS layer for user identification by tying the client to a profile document on the Web through placing a URI in a certificate. It is a first step to a fully standard-based browser authentication experience, but not limited to browser based authentication: peer to peer server authentication will work just as well. The Incubator Group intends to pursue work that has been evolving since 2008, grow the number of interested parties from the Social Web, security and browser communities, and integrate their feedback."
pubsubhubbub - A simple, open, web-hook-based pubsub protocol & open source reference implementation. - Google Project Hosting - Installation guide: PubSubHubbub Support for Drupal | Development Seed. Brand new: Views RSS: PubSubHubbub Elements | drupal.org. FOr pushing 6 and 7: Pusher | drupal.org
Activity Streams Protocol: Hadn't really heard about this one. Looks interesting. "The Activity Streams format has already been adopted by BBC, Gnip, Google Buzz Gowalla, IBM, MySpace, Opera, Socialcast, Superfeedr, TypePad,Windows Live, YIID, and many others." activitystreams's Profile - GitHub .
Salmon for the win: Salmon Protocol // See Protocol Summary - Salmon Protocol.- "As updates and content flow in real time around the Web, conversations around the content are becoming increasingly fragmented into individual silos. Salmon aims to define a standard protocol for comments and annotations to swim upstream to original update sources -- and spawn more commentary in a virtuous cycle. It's open, decentralized, abuse resistant, and user centric. (See latest news.) See Protocol Summary for a high level description, the FAQ for common questions and answers, and the Magic Signatures and Salmon Protocol specifications for full details. There is a Google Code project for source code and specs, and a mailing list; please join the list if you are interested in participating in defining or implementing the protocol. There is a demo running at the Salmon Playground."
OccupyTalk.org and Mumble, JSON hotness: another development is open source Mumble based OccupyTalk. Mumble software kind of annoying to setup so I haven't messed around on it. but good stuff.
It's now possible to embed the Mumble block on your website! This technology is called Channel Viewer Protocol. In short CVP.
Currently we support 1 way of embedding the block: Json CVP.
The address for the Json output is: http://occupytalk.org/json/1/1
Plugins for your website that work with Json:
Drupal 7 module: download
Wordpress plugin: Website
Joomla Channel viewer: Website
More links: Federated Social Web | Drupal Groups. Just a stub: sanduhrs's sandbox: OStatus | drupal.org // RSS & Aggregation | Drupal Groups // Google Feed API — Google Developers // funky Ajax techniques 25 Excellent Ajax Techniques and Examples
Some neat geographic data mapping: I wanted to stuff some more not-yet-fully-baked module links in here, why not. For cool spatial portals, some of this stuff could work wonders. OpenLayers its own badass system for open source mapping. This mapping of river level data, live, is AWESOMESAUCE: AustroFeedr | Hochwassernachrichten in Echtzeit. A neat aggregator of data: AustroFeedr | Pushing the Real Time Web in Austria. English writeup is VERY interesting: AustroFeedr - Real-time Open Data Aggregation, Processing and Publishing in Austria | drupal.org // developmentseed/openlayers_themes - GitHub // Diff | drupal.org < adds cool wiki style changelog effect to revisions. //Search API Location | drupal.org // affinitybridge/spatial - GitHub - php processing spatial data recently developed //Geofield | drupal.org
Entity API relation : Not going to get into it too much, but a lot of cool data can be aggregated and represented as entities rather than nodes in Drupal 7. Anyway these are at varying states but definitely worth looking at for a minute: Relation | drupal.org - cutting edge new way to relate new data entities - specifically designed for linking say donor entries...Entity reference | drupal.org. Hm: References | drupal.org.Field group | drupal.org / Synonyms | drupal.org / DataStore | drupal.org /
WTF: Endgame on this stuff? This post is intended as a rough round-up so hopefully interested folks can find some tasty repos & move this process along :) Basically I think if you mix together a lot of these technologies and make bootable virtual machines and USB images, you will have a ready way to run a lot of great services for almost any circumstance. The AOL-like Facebook monopoly will finally fade away, its primacy eclipsed by obsoleteness. Mix the some of the above open techs, have a few distros moving around the Occupy movement and elsewhere that are more ready-to-run & essentially turnkey, you've got most of what you need without a corporate behavior-spy watching your every move.
In occupy camps, even old cell phones, with no valid SIM for the commercial network, could be talking to a DIY picocell tower and this someday, if I can speculate could be managed by the FGA or other support system. A clever person pointed this out to me. We can dream...
Peer-to-peer CDN networks seem to be one thing needed (SMS gateways another) to replace the most expensive elements of Internet media distribution. Could peer participants in a CDN be awarded rewards much as bitcoin miners might - using Salmon or something else to upstream their available IP connections? Why not? :-D