Groovy and Del.icio.us
Document Actions
|
|
I've been using the collaborative bookmarking app del.icio.us since March. Del.icio.us is a pretty simple ReSTful application, however behind that simplicity is incredible power. It takes a while for this to sink it, but many people are spawning new and compelling applications.
For example, Jon Udell has pumped his blog entries into del.icio.us so he could aggregate all bookmarks related to each entry. Clay Shirky while playing around realized that he could start and maintain conversations using del.icio.us mechanisms.
I expect the emergence of even more applications based on this substrate. However, although del.icio.us helps me find similar and related links, del.icio.us has yet to auto-classify my bookmarks. Anyway, I quickly put together a couple of Groovy scripts, a first step in this journey.
The first script employs the del.icio.us apis:
import org.dom4j.*;
import org.dom4j.io.*
import org.jaxen.dom4j.*;
import org.apache.commons.httpclient.*;
import org.apache.commons.httpclient.methods.*;
username = "user";
password = "pass";
url = "http://del.icio.us/api/posts/get";
creds = new UsernamePasswordCredentials(username, password);
client = new HttpClient();
client.getState().setCredentials(null, null, creds);
method = new GetMethod(url);
client.executeMethod(method);
responseStream = method.getResponseBodyAsStream();
doc = new SAXReader().read(responseStream);
method.releaseConnection()
method.recycle()
print "tags\n";
path = new Dom4jXPath("/tags/tag");
tags = path.selectNodes(doc);
tags.each{ print """${it.attributeValue("tag")} ${it.attributeValue("count")}\n""" };
The code uses Dom4j, Jaxen and HTTPClient libraries. While I was coding this I realized that I didn't want to slam the del.icio.us server too often. So, I wrote a script that parsed the downloaded XML version of my bookmarks. You can massage this script a bit so that it'll export into a format of your choosing. For example, the Firefox bookmark format or XBEL.
import org.dom4j.*;
import org.dom4j.io.*
import org.jaxen.dom4j.*;
filename = "recent.xml"
doc = new SAXReader().read(filename);
path = new Dom4jXPath("/posts/post");
tags = path.selectNodes(doc);
tags.each{ print """${it.attributeValue("href")}
${it.attributeValue("description")}
${it.attributeValue("hash")}
${it.attributeValue("tag")}
${it.attributeValue("time")}\n
""" };
However, I quickly realized the limitations of the del.icio.us APIs. The problem is that you can only access your own bookmarks, you can't see other people's bookmarks! The way around that is to simply go against the html pages. So, I wrote another script that did just that, it exploits TagSoup to handle problems with HTML.
import org.dom4j.*;
import org.dom4j.io.*
import org.jaxen.dom4j.*;
import org.apache.commons.httpclient.*;
import org.apache.commons.httpclient.methods.*;
name = "popular";
url = "http://del.icio.us/${name}";
client = new HttpClient();
method = new GetMethod(url);
client.executeMethod(method)
doc = new SAXReader("org.ccil.cowan.tagsoup.Parser").read(method.getResponseBodyAsStream());
method.releaseConnection()
method.recycle()
path = new Dom4jXPath("//h:div/h:a[@class='delLink']");
path.addNamespace("h","http://www.w3.org/1999/xhtml");
subpath = new Dom4jXPath("h:a[@class='delTag']")
subpath.addNamespace("h","http://www.w3.org/1999/xhtml");
links = path.selectNodes(doc);
links.each { | link |
println link.attributeValue("href")
println link.text
tags = subpath.selectNodes(link.parent)
tags.each{ println "\t ${it.text} ${it.attributeValue("href")}" }
println ""
}
Finally, in del.icio.us you can't find out who bookmarked a URL unless you add it to del.icio.us (or find someone else who did). To do this without creating a new bookmark you can use the following script:
import org.dom4j.*;
import org.dom4j.io.*
import org.jaxen.dom4j.*;
import org.apache.commons.httpclient.*;
import org.apache.commons.httpclient.methods.*;
import java.security.*;
def hash( url )
{
md = MessageDigest.getInstance("MD5");
md.update(url.getBytes());
digest = md.digest();
i = 0;
hash = "";
while ( i < digest.size()) {
String hex = Integer.toHexString((int)digest[i]);
if (hex.length() == 1) hex = "0" + hex;
hex=hex.substring(hex.length()-2);
hash += hex;
i++
}
return hash;
}
name = "http://www.quong.com/shellin20/";
url = "http://del.icio.us/url/${hash(name)}";
client = new HttpClient();
method = new GetMethod(url);
client.executeMethod(method)
doc = new SAXReader("org.ccil.cowan.tagsoup.Parser").read(method.getResponseBodyAsStream());
method.releaseConnection()
method.recycle()
path = new Dom4jXPath("//h:div[@class='delPost']");
path.addNamespace("h","http://www.w3.org/1999/xhtml");
links = path.selectNodes(doc);
println "number of bookmarks: ${links.size()}"
path = new Dom4jXPath("//h:div[@class='delPostExtended']");
path.addNamespace("h","http://www.w3.org/1999/xhtml");
comments = path.selectNodes(doc);
println "comments"
comments.each{ println it.text }
This is a nice starting point. There is one additional method of accessing del.icio.us, that is via RSS. However, that information is a subset of what is available in the APIs, the XML download and the HTML pages. In the future, I'll show you how to put together a nice Groovy abstraction layer to make life even easier.
