Disagree was a bit too strong of wording; rather, if you can watch your users, you should absolutely take the chance to because it's even better than talking to them.
We build enterprise software, so it's a easier for us to go about this.
When we first started doing UX sessions, we created a few scripts that go through basic functionality, but soon found those to be too far-reaching and time-consuming.
Now, whenever we are creating a new feature or testing a new layout, we invite several customers — particularly any who requested said feature or complained about the layout — to test drive it on an internal server, sometimes with a copy of their database if they have specific use-cases, during a screensharing session. Generally, we'll pick 3-5 customers, and do a 1 hour session each. Our development team as well as the sales team watch the sessions take notes, while our project manager guides the client through a script. We take all the feedback, triage it, implement any changes, and then invite them back to test drive again before rolling it out.
We're still learning ourselves, but found the advice in this blog to be really beneficial to the way we conduct our sessions.
We're a place where coders share, stay up-to-date and grow their careers.
We strive for transparency and don't collect excess data.