What's the best way to get started with machine learning?

If you have the time and the love for online course, I highly recommend Udacity's Intro to Machine Learning. I think this is a great way to get a high-level understanding without digging too deep into the mathematics behind the algorithm.

Obviously I think my Machine learning textbook is pretty good :) However, the current version is admittedly rather hard for beginners. I am actually in the process of writing the second edition, which will ramp up more slowly, making it more accessible to beginners. (I'm also adding new content on deep learning, reinforcement learning, etc.) But it will take me a while to finish (~2 years?).
You can also learn machine learning here: hackr.io/tutorials/learn-machine-l...

In the meantime, there are many good books available. See eg this list: josephmisiti/awesome-machine-learning . One book which I think is particularly good for beginners is Introduction to Statistical Learning, by James, Witten, Hastie and Tibshirani. It has a few references to concepts from frequentist statistics, such as p-values (which you can safely ignore :), and doesn't cover topics such as deep learning or graphical models, but nevertheless it's a good place to start.

Classifying documents, or websites, can be an interesting way to start. Start with a list of domains that you like on a topic, and a bunch of unrelated domains. Now try to build a system that will pick out the other domains you like. This will introduce several of the concepts behind machine learning:

  • attribute extraction: picking data out of the websites for use as attributes (this can be as simple as parsing the HTML and pulling out words)
  • training: using known good data (a subset of the domains you like) to train the system
  • classification: let the system run on the new data. You have only two categories at this point so it should be easy to manually confirm the results.

I'm specifically not including any technologies or programming languages here. All languages have numerous libraries, and just following the concept words, or just tracing machine learning should pick out many. But the above gives the basic outline of what you're trying to do, to help guide the search.

(Note, this basic task doesn't involve neural networks. Don't get side-tracked, at least not yet at least.)

Once you get into it you'll start learning about the statistical models being used, and can start branching off. You can look at classifying more categories. Identifying attributes automatically. Correlating documents to each other. Pattern and behavior prediction.