There is confusion out there between these three terms. The DOM is an absolutely integral portion of modern web development, and knowing how to differentiate the three and knowing when to use these is crucial for being an effective web developer.
So when you have markup on your page with HTML, your browser compiles all that into a Document Object Model which we abbreviate to the DOM. This is an application programming interface (API) that the browser gives you in order to be able to interact with the contents that are rendered on the page.
This sounds more complicated than it is, because at its simplest, you can throw an html file into a server and what will come out are those contents rendered, just like you wrote them. You can think of the DOM as the way that the html is presented to you when you inspect your browser window and go to the elements tab.
This allows many different and cool things for web applications. Not having to worry about rerendering every time that something is changed, allows frameworks like react to update only the nodes which have changed when the data gets updated. These diffing algorithms are extremely powerful and make these frameworks very effective ( ...and sometimes fast!).
The Shadow DOM is something completely different than the virtual DOM. The shadow DOM is a browser specification for building completely isolated HTML elements that are not affected by the styles that are currently affecting the rest of the page. It is one of the specifications that are aiming at making modular HTML components which can be imported into and out of different documents.
I'd say that this is not so popular in enterprise applications, but it allows for building HTML without the overhead of frameworks through the Web Component Specification. It can be tricky if you encounter it and don't know how to use it, but very helpful if you use it deliberately.
Hope that this helps you out if you are having trouble figuring out the differences.