Yeah I thought about using CloudFlare but only the traffic between the user and CloudFlare is encrypted. Traffic between GitHub and CloudFlare will be plain text. There's probably still an advantage, however I don't like the idea of making the user think their traffic is encrypted end-to-end when it's actually only encrypted half way. If my site transferred any sensitive information, I definitely would have searched for another solution.
Though it might be worth it to look into CloudFlare for the other benefits it can provide!