what are Gestures anyway?

theoldman — Tue, 01 Nov 2022 16:28:50 +0000

Boring content alert, Not coool stuff.

So I was playing around with webShare API (the reason being now it supports desktop as well, it was only available in mobile when I last checked.) (navigator.share), and one interesting thing I found was you can not call navigator.share randomly anywhere, you need a "user gesture (that's what the error said )", I thought maybe scroll could be used to abuse the navigator.share and various other APIs that need user action, or rather I should say user gesture.

Of course, if you think about it, it should be very natural to think that things like scroll and hover should be considered user gestures, at least that's what I used to think, and things like navigator.share() should work when using them on hover right?

If this was the case I would have done something like this,

// bool shareOpened = false; 

let shareOpened = false;

document.body.addEventListener("mouseover", function () {
    if (!shareOpened && "share" in navigator) {
        navigator.share({
            data: "spam",
            url: "https://spam_link.com"
        }).then(val => {
            console.log("ok");
            shareOpened = true;
        }).catch(e => {
            console.log(e);
            console.log("!ok");
        })
    } else {
        shareOpened = true;
    }
})

Now, I'm not a spammer I don't know what someone would achieve by banging the web share API, but as a developer, I can tell you you will get exceptions like this

DOMException: Failed to execute 'share' on 'Navigator': Must be handling a user gesture to perform a share request. at HTMLBodyElement.<anonymous> (https://my_shit_code.com/script.js:17:19)

Now I want you to try something -

Click anywhere in the screen, hover out of the screen and hover again, to see if this works now, you don't need a gesture now, it works with just hovering.

This is crazy right, some of you might even try to open the chromium bug website to report it there, at least that is what I was about to do, but no it's not a bug this is how chrome user gestures work.

So today I would like to share some stuff with you.

To prevent the abuse of APIs like this, chromium based browsers ( I don't know about firefox ) have come up with some solutions, the solution is the separation of user activations and user-gesture

Almost all things are user activations (scroll, hover, and various others and spammers use these things to abuse the APIs).

There are only four (4) things that are considered user gestures, and here is how chrome has implemented it

bool IsUserInteractionInputType(blink::WebInputEvent::Type type) {
  // Ideally, this list would be based more off of
  // https://whatwg.org/C/interaction.html#triggered-by-user-activation.
  return type == blink::WebInputEvent::Type::kMouseDown ||
         type == blink::WebInputEvent::Type::kGestureScrollBegin ||
         type == blink::WebInputEvent::Type::kTouchStart ||
         type == blink::WebInputEvent::Type::kRawKeyDown;
}

Once a single gesture is fired many APIs allow running the things like navigator.share without any gesture, that is what happened in the above code, and this is only allowed to do for 5 seconds after 5 seconds (time may differ, I'm saying based on my research ) it will not work. so let's try running our code once again by modifying a little

document.body.addEventListener("mouseover", function () {
    if ("share" in navigator) {
        navigator.share({
            data: "spam",
            url: "https://spam_link.com"
        }).then(val => {
            console.log("ok", val);
        }).catch(e => {
            console.log("ok!", e);
        })
    }else{
        console.log("not supported");
    }
})

If I run the above code on a fresh page, navigator.share will not open but if I click anywhere the user gesture will be registered and for the next 5 seconds my code will work without any gesture, as soon as I will hover the popup for share will open.

There are also things called consumable gestures, which means they require user gestures for everything. The 5-second thing doesn't work on those calls, so you have no way to abuse these APIs, you will have to rely on a user gesture.

I myself haven't read all the details yet, so I will not pretend to know everything, but if you want more on this topic check the references below. Things are not very simple, it needs a thorough reading, in my opinion, to understand in detail.

Here is a simple youtube video https://youtu.be/r1z2R3qX9uQ (sorry Reddit takes forever to upload ) without audio (sorry), as you can see first it did not work but once I clicked it started to show the share popup, and again it expired after about 5 seconds, you can try it yourself.

Code I used

<!DOCTYPE html>
<html lang="en">
<head>
    <meta charset="UTF-8">
    <meta http-equiv="X-UA-Compatible" content="IE=edge">
    <meta name="viewport" content="width=device-width, initial-scale=1.0">
    <title>Web Share</title>
</head>
<body style="height:100vh;">
    <button>Share this page, just for nothing</button>
    <script>

document.body.addEventListener("mouseover", function () {
    if ("share" in navigator) {
        navigator.share({
            data: "spam",
            url: "https://spam_link.com"
        }).then(val => {
            console.log("ok", val);
        }).catch(e => {
            console.log("ok!", e);
        })
    }else{
        console.log("not supported");
    }
})

    </script>
</body>
</html>

References and tools that might help you

Chrome Code - https://source.chromium.org/chromium/chromium/src/+/main:content/browser/web_contents/web_contents_impl.cc;l=266;drc=9d82515060b9b75fa941986f5db7390299669ef1
If anyone is interested in reading more boring content - https://html.spec.whatwg.org/multipage/interaction.html#triggered-by-user-activation
https://mustaqahmed.github.io/user-activation-v2/ (user activation)
https://bayden.com/test/popup/#timers ( a tool some nice person made for timers - the 5s thingy )
BTW Here is the list of APIs that need user gesture - Google Docs Document

DEV Community: theoldman

what are Gestures anyway?

Now I want you to try something -

Code I used

References and tools that might help you