Mobile/Display Interaction using Registered Displays

Nick Pears, Patrick Olivier, Dan Jackson, Leigh Herbert

Home | Research projects | Publications | Teaching | Short bio

In this project, we use a camera-equipped mobile phone to view a PC screen (or public display) and we can control many interactions between the two devices. The essence of the idea is that, after estimating the planar projective mapping (homography) between the image on a mobile and a (possibly distant) display, any 'click' on the mobile's touch screen can be projected onto the correct corresponding point on the distant display. Of course, a wireless communication link is required so that the mobile can send its image to the display's PC, along with the point 'clicked' on its touch screen. Our idea has been reportred in a 'New Scientist' article in January 2008. This report made the top ten articles in the ACM TechNews alert, and it has been reported in many press articles and on many websites around the world.

Video demonstrations

Publications.

The key publication is number 2 below. Paper 1 describes a new markerless registration scheme, based on scale invariant features such as SURF and SIFT. Paper 3 is the original publication on the `display registration' idea.

  1. Mobile Device and Intelligent Display Interaction via Scale-Invariant Image Feature Matching.
      L. Herbert, N. E. Pears, D. Jackson and P. Olivier
      1st Int. Conf. Pervasive and Embedded Computing and Communication Systems (PECCS 2011)
      [PDF] [BibTeX]

  2. Smartphone Interaction with Registered Displays
      N. E. Pears, P. Olivier and D. Jackson.
      IEEE Pervasive Computing, vol 8, no 2 (April-June 2009), pp 14-21.
      Special issue on Smarter Phones.
      [PDF] [DOI][BibTeX]

  3. Display Registration for Device Interaction
      N. E. Pears, P. L. Olivier and D. Jackson
      3rd Int. Conf. on Computer Vision Theory and Applications 2008 (VISAPP'08), 22 - 25 March, 2008, Funchal, Madeira, Portugal.
      [PDF] [BibTeX]


BACK to Nick Pears' Research Projects page.