• Canoe 2009 Applications

What are the compelling pub/sub applications, and how can we promote event-based application development?

Participants: Vinod Muthusamy, Patrick Lee, Mo Sadoghi, Michael Olson, Darko Anicic, Amer Farroukh

Guiding Questions

Here are some questions to get you started, but feel free to define your own.

• How can we bring up a world-wide pub/sub network or event processing system?
• What applications would drive the use, participation, and adoption of such a system?
• Can we run a generic event processing system playground across dedicated nodes?
• Who would/should the users be?
  • Pub/Sub researchers, other researchers, application developers, end-users?
• What are killer applications for distributed event-based systems?
  • How can event-based systems help build global sensor networks?
  • How can small-scale actuator/sensor networks (e.g., in an e-home scenario) profit from pub/sub?
• What apps are the low hanging fruit for adoption?

  • What applications today are naturally modeled with events, are easy to implement, are widely used, would benefit from events & pub/sub and EP?

Summary of breakout session discussion

As our group was composed of students, our discussions tended to revolve around applications we would use personally. That is, we focused on consumer applications, as opposed to enterprise or industrial applications.

What are the most useful features of pub/sub in consumer applications?

• push

• fine-grained filtering

• decoupling (debatable)
• multicast (more useful to the infrastructure)

Personal information broker

One class of applications is personal information management.

We consume a lot of information:

• email
• blogs
• news
• instant messages
• friend status
• reminders
• twitter feeds
• product promotions
• flight status
• etc.

What would be useful is a single broker that would act as

• a central receving point for all communication from outside
• a central sending point for all communication to the outside

Open address book

An example of one functionality of this personal information broker is an updatable address book.

• When my friend moves, they email me their new phone number, address, etc.
• I have to update my phone contact (but only after they've moved).
• But I shouldn't have to know that they've got a new phone number.
• Contact updates should be pushed and updated without me knowing.

How to deploy this?

• Google can do it in Gmail.
• Poll user's website for updates to their vcard. (not push)
• Develop applications to publish profile updates, subscribe to updates; and deploy a broker. Is it worthwhile?

Matching classifieds, goods

Other useful services would be to be notified when

• An apartment listing on Craigslist appears that satisfies my criteria.
• A book of interest on Amazon goes on sale.
• A flight to a specific destination is available for a cheap price.

Developing such services by a third party will require extracting this information (scraping sites, or using provided APIs).

Twitter

Publish twitter feeds, and allow users to subcribe to them.

Health records

Transfer a user's health information to whatever city they travel to. Does this require push?

Continuous web search

As of two days ago, Google Alerts are can be subscribed to through PubSubHubbub.

What's wrong with polling?

For a lot of the data that is of interest to an end user, high latency seems tolerable.

• e.g., email, (most) news, (most?) reminders, etc.

For this data, polling is acceptable.

The problem with polling is its inefficient use of network resources.

BUT, can push communication be less efficient than polling?

• Push requires registering filters.

• These filters are evaluated for every event, even if the user is cannot respond to the event at the moment (i.e., user's attention is elsewhere).

• Polling on the other hand, only consumes resources when the user is ready to receive data.

So, for low update data streams and subscribers with limited attention spans, can pull scale better than push?

When to use push?

Data pushed to a user should require their immediate attention. Only high priority data should be pushed.

This requires:

• precise filtering
• context-awareness

Filtering

It is not always easy for users to specify their interests.

• Whose facebook posts do I really want to follow?
• Which news items do I want to read?

Many applications can use machine learning techniques to discover a user's interests. Is machine learning an orthogonal problem to event processing?

Context-awareness

It is natural for a user's interests to change with their context.

• Don't disturb me when I'm in a movie theater.
• Block instant messaging when I'm giving a presentation.
• Don't alert me about job listings when I'm at work.
• Tell me about relevant promotions/sales when I'm shopping.
• Only accept emergency calls when I'm in a meeting with my boss.
• Notify me of local events when I'm on vacation.

There are many ways to determine a user's context.

• explicitly by the user
• wifi signals
• gps location, speed
• time of day
• appointments in calendar
• open applications
• etc.

A user's personal information broker should be context-aware. For example, when I arrive in Oslo, my interests include

• local health alerts
• status of Facebook friends, other contacts in the area who are available to meet
• promotions by restaurants of a certain style (and within some price range)
• etc.

How should context be integrated with a user's interests (i.e., subscriptions, rules, etc.)?

• Is context state? Or is it a history of events? Or both?
• Should a user's subscriptions change along with their context?
• Should a subscription language be able to express context constraints?

What is a subscriber's endpoint?

How to push information to a user?

• Post to a user's web server. (See PubSubHubbub.)

• Long polling using long-lived TCP connection.
• Send an SMS message.
• ???

Bringing up a pub/sub network

One approach is to exploit Google PubSubHubbub.

• Publishers publish an RSS feed.
• Subscribers subscribe to a feed (topic-based).

Can we build filtering using this? One possible way is as follows:

• Deploy a content-based broker (network) that reads from a publisher's topic and writes to a subscriber-specific topic.

• In this way, PubSubHubbub only acts as a Web-based interface to the publishers and subscribers.

Other resources

DEBS keynote on Event-Based Applications and Enabling Technologies

Google PubSubHubbub