I find it extremely annoying, nor do I understand overall the real intent or use of it. I make an exception for things that are audio-dominant--ie, a podcast, music videos, long videos that are really just a music playlist, etc.
If the video is genuinely key to the content and page, then that should be largely all that's there (barring other interface things and stuff like comments, etc, of course), and there should be no need for it to "follow" the user like that as there shouldn't be anything so inaccessible from the viewport of the video so as to detract. Things like attribution, source, etc., are fine, of course, but if there is information the user needs to be accessing while watching the video, it should be set up so that is already visible.
If it isn't that key to the content and page, then it definitely shouldn't follow the user.
I could think of a few contexts where it would be actually useful--say on a tutorial page for assembling something or something similar, where you can keep the video going but also scroll to check a parts list or refer back to a diagram or earlier step. Even then, I'm not sure it's that helpful, since the video gets small and you can't really attend to it and whatever you're looking at on the page simultaneously.
What I would find useful would be--assuming the video is relevant and key to the page and genuinely the right format for the content (which they often aren't, but that's a soapbox for another day)--is something where on scroll, the video moved aside and paused. I feel like that respects the fact that the user is scrolling for some actual reason while still highlighting the video.
We're a place where coders share, stay up-to-date and grow their careers.
We strive for transparency and don't collect excess data.