Sri Lankan Incident Mashup

I just posted a finished version of a Mashup designed to help answer the question "is Sri Lanka getting safer or not?"  This is a question we on the global WSO2 team ask each time we arrange travel to that unfortunately troubled country.  Despite a spate of violence early this year, designed to coincide with the formal dissolution of the cease fire that has done little to prevent violence, it seemed to me things were getting a little better.  But I needed some facts to back that up.

image The mashup plots bombings and other incidents as a bar chart, measuring the severity of the incident (how many killed & wounded) over the last 18 months.  The idea was to see if there was a clear overall trend in the violence or not, something not readily apparent from a Google map (see right).

The mashup service itself consists of several items, each one a simple task accomplished in half a page of javascript:

  • Scrape the search results page at globalincidentmap.com to pull the essential data out of an HTML table for the country of interest, and put it into a simple XML structure.
  • Cache the page if it has already been accessed within 24 hours (scraping is expensive, the first access in any 24 hour period will be pretty slow.)
  • Parse the headline for patterns such as "eight killed" or "injures 7" and turn that into killed and wounded digits (this isn’t perfect, but we can tolerate a few errors since we’re trying to present an overall sense of the problem rather than perfectly accurate statistics.)  Also filter out as much as possible the killing of LTTE, as that’s more a measure of war than of terrorism.
  • Provide a helper method to look up details and get a link to a new story for any item.

Using this service (called internationalindicent) I created a custom HTML UI to format Sri Lanka-specific results into the bar chart and to make it interactive (click on a bar to see more info about the incident.)  Then I used "share this mashup" to upload the service to mooshup.com so others could try it out (or copy the code.)

The rough version took a couple of hours, mostly figuring out the details of scraping the page and coming up with the headline patterns to look for, but then I spent a couple more polishing the HTML UI so I wouldn’t be embarrassed to share it.  In the process I demonstrated some of the powerful aspects of using mashup technologies in your development arsenal:

  • Instead of investigating the current situation using Google News for 30-60 minutes each time we plan a trip for the latest information, I can browse this site in a few minutes, see the trend, and get details of any recent incidents I’m interested in.  This will pay for itself in terms of my own personal productivity before long.
  • There is also a small user group (really, just the handful of WSO2 employees based outside Sri Lanka) who can also benefit from this micro-application, increasing their productivity as well.
  • The WSO2 Mashup Server makes the data available as a service, so others can reuse it too, for alternate displays or to generate displays for other parts of the world.
  • And, it’s just fun!

So … is Sri Lanka getting safer?  I’ll have to let you be the judge of that.  Go to http://mooshup.com/services/jonathan/internationalincident/ to see for yourself!

[Update 5/27: This service is no longer available.  See the newer entry for more info.]

Mashup Camp 6

20080319-_ND32514Just returned from some travel which included 2 days of Mashup University and a day of Mashup Camp.  A few thoughts:

  • Maps are still well represented.  This surprised me a bit, but I recognize I’ve been well immersed in the WSO2 Mashup Server which supports a wide variety of User Interactions (html pages including maps, but also feeds, email, instant messaging.)  I predict non-map mashups will start to eat into the map-based mashup market share dominance over the next year.
  • There’s lots of interest in consuming various Web APIs.  Some vendors were promoting their APIs, others tools for consuming those APIs.  I think the WSO2 Mashup Server can tap into an underserved market here, since it’s the easiest way I know of to deliver a comprehensive Web API on top of a bit of Javascript logic, which in turn can front information sources as diverse as databases and scraped web pages.  I think the Mashup Server can become a "design your own API" tool that can have great appeal to the mashup developer.
  • A lot of Javascript was in evidence, further validating our choice to use Javascript for mashup logic in the WSO2 Mashup Server.
  • For the first time I ran into a few people with serious interest in WADL.  Has its time finally come?  More on that in a subsequent post.
  • The "unconference" style was quite interesting and successful, especially if you’re like me and are more interesting in connecting with interesting individuals than in hearing yet another vendor pitch (mine excepted of course!) ;-)  One thing that is sorely lacking is any kind of organization to the conference Wiki, and a surprising lack of ability to record much of the activity there.  I couldn’t even find who won the Best Mashup contest…

I can’t wait till next year!