====== DokuWiki Publish/Subscribe ======

Ideas on Server to Server communication to synchronize [[doku>|dokuwiki]] content.

===== News =====

  * took the detour to implement an [[dokuoauth|oAuth plugin]]. --- //22.10.2008 13:00//
  * added a development [[http://gareus.org/trac/dokuwikihacks/roadmap|roadmap]]. --- //01.09.2008 12:30//
  * coffee break - installed a [[http://gareus.org/trac/dokuwikihacks|trac for this project]].  --- //31.08.2008 15:18// 
  * **Just** received and processed(!) the first XMPP notification. dokupubsub alpha-1 is out. --- //31.08.2008 13:53// 
  * I've started work on a prototype DokuWiki plugin. Download a [[http://gareus.org/gitweb/?p=dokupubsub.git;a=snapshot;h=head|devel-snapshot]]. dokupubsub interacts with [[idavoll]] to interface to the XMPP world. --- //29.08.2008 15:02//

===== dokupubsub =====
{{:wiki:dwpubsub.png?111  }}
The plugin code is still flux. You can test the [[http://gareus.org/gitweb/?p=dokupubsub.git;a=snapshot;h=alpha-2|alpha devel snapshot]] or have a look using the [[http://gareus.org/trac/dokuwikihacks/browser|source-code-browser]]. You can check out the code with <code>git clone git://gareus.org/dokupubsub</code>//Note//: This plugin is **not safe** to run on any **internet-connected** site just yet!

Additional to installing the plugin into DokuWiki's //lib/plugins// folder, you'll need to run //jabberd2// and [[idavoll|idavoll (setup notes)]] on ''localhost'' with the webserver:

  sudo twistd idavoll-http --jid=localhost
  tail -f twistd.log

The dokupubsub plugin writes debug info to ''/tmp/PubSub.debug''. Subscriptions (automatic wiki-page update notifications) are restricted((hardcoded prefix - during development, )) to the ''ext:'' namespace - adjust the //ACL// accordingly. Currently I'm testing subscribing to publications on localhost using the same DokuWiki.

After editing a page (with the web-interface) it will be published to the gateway; the xmpp uri of the publication is saved with meta-data and added to the HTML header as ''<link rel="alternate feed" href="xmpp:.."/>'' for discovery. Notifications from the gateway are handled with a custom ''do[dokubookmark]=notify'' RPC handler inside the plugin.

There is no PubSub admin interface yet. To subscribe to changes you'll need to make a request specifying the originating XMPP uri and the destination wiki-ID eg:
  curl -d "what=ever" "http://localhost/dokuwiki/doku.php?do%5Bdokupubsub%5D=subscribe&id=wiki%3Astart&nodeid=xmpp%3Alocalhost%3F%3Bnode%3Dgeneric%2Fbe096541-c728-42f8-b0f6-f330a86b5ec7" 
or equivalently:
  curl -d "nodeid=xmpp:localhost?node=generic/123456789-abcd" -d "wikiid=start" "http://localhost/dokuwiki/doku.php?do%5Bdokupubsub%5D=subscribe"

example xmpp-url discovery - see ''/tmp/PubSub.debug''
  curl -d "url=http://localhost/dokuwiki/start"  "http://localhost/dokuwiki/doku.php?do%5Bdokupubsub%5D=discover" 

For testing, you can //fake// notifications by saving an Atom-entry document as file ''/tmp/ps.xml'' and run
  curl  -d "@/tmp/ps.xml" "http://localhost/dokuwiki/doku.php?do%5Bdokupubsub%5D=notify&id=wiki%3Atest" 
Change the ''id=??'' to match the wiki-page-id on the subscribing site.

 

===== earlier ideas and brainstorm =====

==== About ====

Motivation ranges from adding features such as DokuWiki mirror, backup and decentralized load-balancing solutions to semantic web communication and data-portability.

Kludge solutions (like [[dokursync]]) work fine for smaller sites but there's need for improvement on scalability and offering web-standards.

**Outline**

  * http://OAuth.net & http://openID.org
  * RPC
  * XMPP & Atom 

I envisage a subscription and a publication plugin for DokuWiki, making use of existing infrastructure (eg. Atom and raw data export). A mechanism to distribute locks may be included in either or will be implemented in an optional standalone plugin.

Current ideas include using a XMPP <-> HTTP gateway and or the [[http://www.dokuwiki.org/devel:xmlrpc|WikiRPCInterface2]]. Then again, using a custom RPC notification to distribute edit-locks would be easier to maintain, setup and may have lower latency, but lacks interoperability.

The intention is to minimize DokuWiki changes, and also keep the DokuWiki plugins rather simple. re-use existing frameworks and protocols.

**Ideas**

The preferred mechanism would be to implement Jabber XMPP  [[http://www.xmpp.org/extensions/xep-0060.html|Publish-Subscribe]] extension in conjunction with a XMPP -> RPC gatewaty.

Having DokuWiki publish news is a minor task; email-subscription is already possible. Formatting it in XML or Atom and signing the change is straight-forward; wrapping and sending it to a gateway also easy.

Acting upon subscriptions is a bit trickier. It's feasible to just generate an automatic edit request via HTTP-POST for each incoming XMPP news. This could be built-in or happen at a gateway; with design depending on implementation: Authentication could happen internally (DokuWiki signs data-changeset), as part of the protocol (oAuth) or via transport (HTTPS). the choice depends on the use-case and application: user-administrated vs. admin-maintained or low-traffic-decentraliced-wiki vs. mirror/load-balancing.

options: 
  - DokuWiki could use OAuth for setUID-requests - //normal// ''do=save'' POST can be used to perform updates. (oAuth-tokens are tunneled with XMPP-message/request to //any// gateway - openID can be used as //remote// user-account) 
  - whitelist/blacklist approach: local xmpp gateway filters and performs wiki updates (write files or //hardcoded-auth// POST)
  - custom DokuWiki shared-Auth mechanism. (eg. content-signature)

(1) is the most flexible and cool, alas also the most complex choice. (2) is pragmatic and probably fast (good performance and short implementation-time) wheres (3) may offer more advanced SPAM protection, user configuration and mapping and as well be on the way to provide a lock mechanism.

PubSub does not strictly address locking. However Publishing is an atomic transaction in XMPP so a solution would be to publish an "invalidated" version of a page when beginning to edit and re-publish afterwards. It would either require very short draft-timeout times or DokuWiki would need send AJAX requests on cancel edit and re-publish.

A different approach for handling subscriptions would be a standalone gateway that interfaces with DokuWiki over XML-RPC instead of being implemented as as DokuWiki plugin..

==== Resources ====

  * http://wokkel.ik.nu/
  * http://idavoll.ik.nu
  * http://www.dokuwiki.org/devel:xmlrpc
  * http://www.jspwiki.org/wiki/WikiRPCInterface2
  * http://www.mediamatic.net/page/26605/en
  * http://www.mediamatic.net/module/PubSub/
  * http://svn.automattic.com/wordpress/trunk/wp-app.php
  * http://tools.ietf.org/html/rfc5023
  * http://www.tbray.org/ape/

wokkel implements a HTTP <-> XMPP gatway that distributing Atom feeds along with an //authoritative resource URI//. It's intended to run on a host reachable by the web-server (localhost, or in DMZ);  the XMPP port is open to public (it's a jabber server) while the HTTP has a whitelist IP filter.

see http://idavoll.ik.nu/wiki/HTTP_Interface and http://idavoll.ik.nu/wiki/HTTP_Interface_Subscription
After publication the XMPP-node ID should be included in the xhtml header to provide subscription discovery.


as for DokuWiki clustering:
  * http://dokuwiki.org/tips:farm
  * http://dokuwiki.org/tips:farm2
  * http://www.content-space.de/dokuwiki/projekte/dokufs
  * http://chimeric.de/projects/dokuwiki/dokuvimki
  * http://linuxboard.org.ua/notefinder/doku.php?id=screenshots