MeeGo Touch: Gestures and Multitouch

Gestures

A gesture is a high-level event that represents a series of user input. Qt and MeeGo Touch support the following gestures:

Gesture	User interaction	Class
Pan	Press, drag, release	QPanGesture
Pinch	Press (with two fingers), move fingers on surface, optionally release one of the fingers and reposition, release	QPinchGesture
Swipe	Press, quick drag and release	QSwipeGesture
Tap and hold	Press, wait	QTapAndHoldGesture
Tap	Press, release	QTapGesture

As a gesture is merely an interpreted series of input, what the gesture actually means is up to the application. For example, a pinch gesture may be used to either rotate or zoom a picture, or both. If combined, gestures may have overlapping interactions, as is the case of swipe and pan. It is up to the application design to make sure there are no overlapping interactions in UI (the result of overlapping interactions is undefined). Typically a swipe gesture would be used in a UI where a flick should be interpreted as "next" or "previous", while a pan gesture implies an event with acceleration such as scrolling a web page.

While all of the above gestures are touch activated, gestures can potentially use any type of input such as key events or sensor data. Using the QGestureRecognizer framework, it is possible to register new custom gesture types.

Gestures used by MeeGo Touch

While gesture events are mainly meant for application consumption, the MeeGo Touch UI design guidelines does specify some default actions based on gestures. Unless you as an application developer override the behavior, gestures may be consumed by the framework itself in the following cases:

The MPannableViewport consumes QPanGesture events, most commonly in the case of a pannable MApplicationPage (pages are pannable by default).
Any MWidget with one or more QActions or MActions added (using QGraphicsWidget::addAction) will consume a QTapAndHoldGesture and open the contextual "Object Menu" for the widget when the gesture completes.

Multitouch

While some of the gestures mentioned above like pinch are implemented using multitouch events, by itself multitouch merely refers to the capability to detect several fingers on the screen at once. Multitouch events are also directly accessible by applications in a low-level form comparable to mouse and key presses and releases, where the actual interpretation of the events is left entirely up to you.

Note that multitouch requires special hardware to function, many touch input devices do not support the capability to detect multiple touch points at once.

In Qt, multitouch input is delivered through the QTouchEvent class, through which it is possible to determine all the currently touched screen points. Note that by default multitouch events are not delivered to widgets, delivery must explicitly be enabled using the QGraphicsItem::setAcceptTouchEvents method.

Simulating multitouch

You can simulate the multitouch pinch gesture without needing real multitouch hardware, for example in a development environment on a standard PC. Simply hold down the <Ctrl> key on the keyboard, then press and drag with the left mouse button.

Gestures related API in MeeGo Touch

MeeGo Touch provides several event handlers for gestures:

These event handlers can be re-implemented to act upon all or specific gestures, in the same way as the existing methods QGraphicsItem::mousePressEvent, QGraphicsItem::mouseReleaseEvent and so on.

There are also corresponding methods in both the MWidgetController and the MWidgetView for use when extending or writing new MeeGo Touch common components.

Gestures are not delivered to widgets by default, gesture delivery is enabled by the QGraphicsObject::grabGesture method.

MeeGo Touch currently replaces some of the default Qt gesture recognizers with versions capable of intepreting regular mouse events in addition to the touch events used by Qt. Pan and swipe are also one-finger gestures in MeeGo Touch while they are 2 and 3 finger gestures respectively in Qt (as of version 4.7).

Gestures Example & Tips for Apps

The examples/gestures directory of the MeeGo Touch source contains a sample application demonstrating proper use of gestures combined with MeeGo Touch widgets. The example is of a gallery type of application, where pinching the picture zooms and rotates simultanously, while swiping left or right changes the picture.

Notable points of the example:

Enabling multitouch handling through setAcceptTouchEvents(true);
Enabling gesture delivery through grabGesture
Transforming the picture by calculating the rotation and scale factors using the pinch
The ease of detecing and acting on a swipe gesture
Best practices of detecing the gesture on the largest surface available

Below is the example code relevant to the gestures:

MyPage::MyPage(QGraphicsItem *parent)
    : MApplicationPage(parent),
    currentImageNumber(0)
{
    setTitle("Gestures example");

    // This enables delivery of low-level touch events to the widget.
    // While we aren't directly interested in these, on a multitouch
    // capable system the pinch gesture is recognized by the touch events.
    setAcceptTouchEvents(true);

    // The grabGesture methods enables delivery of the high-level gesture
    // events (QPinchGesture and QSwipeGesture in our case) to the widget.
    //
    // The reason the gestures are handled by the page instead of the
    // actual image widget, is that we want the entire page area to
    // function as a single pinching/rotating/swiping surface.
    //
    // In general, you should handle gestures on the largest possible
    // area, even if it holds multiple widgets. In case of multiple widgets,
    // you can determine the target of the gesture using the event's hotSpot
    // property.
    grabGesture(Qt::PinchGesture);
    grabGesture(Qt::SwipeGesture);

// The gesture handlers:

// Pinch handler
void MyPage::pinchGestureEvent(QGestureEvent *event,
                               QPinchGesture *gesture)
{
    mDebug("pinchGestureEvent")
            << "State:" << gesture->state()
            << "Angle:" << gesture->rotationAngle()
            << "Scale" << gesture->scaleFactor();

    // By accepting the event we stop it from propagating to the
    // next widget.
    event->accept(Qt::PinchGesture);

    // Setting the transformation point to the center of the image.
    // This makes the image rotate around the center when we pinch and rotate,
    // instead of the top-left corner which is the default,
    MImageWidget *currentImage = images[currentImageNumber];
    currentImage->setTransformOriginPoint(currentImage->size().width() / 2,
                                          currentImage->size().height() / 2);

    // Now let's manipulate our image widget according to the data
    // contained in the the pinch gesture.

    // It is important to exclude gesture states other than "updated"
    // from the manipulation of the widget, as for example on
    // Qt::GestureStarted both the angle and scale will be 0.
    if (gesture->state() == Qt::GestureUpdated) {

        // The rotation delta is how much additional rotation we should apply
        // to our image. The absolute rotation value of the gesture is not
        // of interest, applying it to our image would make the widget "jump".
        qreal rotationDelta = gesture->rotationAngle() - gesture->lastRotationAngle();
        qreal newRotation = currentImage->rotation() + rotationDelta;
        currentImage->setRotation(newRotation);

        // Same deal for the scale as above, the current scale of the widget is
        // adjusted while the absolute scale value of the gesture is not important.
        qreal scaleDelta = gesture->scaleFactor() - gesture->lastScaleFactor();
        qreal newScale = currentImage->scale() + scaleDelta;
        // Let's limit the range the image can be scaled: Shrink by 80%, grow by 400%
        if (newScale > 0.2f && newScale < 4.0f)
            currentImage->setScale(newScale);
    }
}

// Swipe handler
void MyPage::swipeGestureEvent(QGestureEvent *event,
                               QSwipeGesture *gesture)
{

    // If the vertical direction of swipe is other than NoDirection
    // then the user did not swipe horizontally enough. We will ignore
    // that event. It will be propagated to the widgets beneath us
    // in case any of them is interested.
    if (gesture->verticalDirection() != QSwipeGesture::NoDirection) {
        event->ignore(gesture);
        return;
    }

    // By accepting the event we stop it from propagating to the
    // next widget.
    event->accept(gesture);

    if (gesture->state() == Qt::GestureStarted) {
        // Swiping towards left, brings in a new picure from the right
        if (gesture->horizontalDirection() == QSwipeGesture::Left) {
            mDebug("swipeGestureEvent") << "Left";
            showNextImage();
        } else if (gesture->horizontalDirection() == QSwipeGesture::Right) {
            mDebug("swipeGestureEvent") << "Right";
            showPreviousImage();
        }
    } else if (gesture->state() == Qt::GestureCanceled) {
        // Revert the transition of images. The user didn't
        // do swipe gesture after all, he could for example change
        // the direction of finger movement.
        if (gesture->horizontalDirection() == QSwipeGesture::Left) {
            mDebug("swipeGestureEvent") << "Cancel left movement";
            showPreviousImage(true);
        } else if (gesture->horizontalDirection() == QSwipeGesture::Right) {
            mDebug("swipeGestureEvent") << "Cancel right movement";
            showNextImage(true);
        }
    }
    // We will also be getting GestureUpdated and GestureFinished states
    // here, but we don't need to react to them here.
}

// End of the gesture handlers