Hello everyone,
a couple of weeks ago I found a site https://jschr.io and I really wanted to build something like this myself. The author wrote a blog post about his way of doing things. But it involves JS, AWS and some more stuff I really don't want to use. So here is my question:
Can anybody help give me advise on how to build a static site with dynamic content like latest toots, blog posts, and some more stuff? I would like to use Shell Scripts or Python and everything should be selfhostable. No cloud stuff.
Thanks for any help :D
Top comments (9)
Hi! I can give some ideas :)
It really depends on a lot of factors.
If you want your latest tweets to be real-time, you'll probably need something more dynamic, since Twitter's API requires a key and you probably don't want that floating around in front-end code, necessarily. If you just want to get stuff from GitHub, you might be OK. In any-case, long-story short, if you want the data to be real-time, you'll either have to have some front-end JavaScript in your HTML that goes out and grabs that data (not as scary as it sounds), or you'll need some sort of server backend that goes and gets it. That server backend can be in Python, which is nice, but it's a little more complicated and not static like you said you wanted.
Now, depending on your blogging frequency or if you're not opposed to having some lag in your data, a static-site generator/scripts should be totally fine. Your build/deploy process can include all of the work to fetch all of the data before it brings it in and renders the HTML. This is my personal favorite approach. I have an unhealthy love for all things that compile static documents from templates, and any time I hear about a new static site generator, I love to try it out. Anyways, there are a few good ones for Python that do a lot for you.
Pelican is probably the most popular and the most similar to static site generators in other languages. There are probably good templates out there and it's probably not too bad to create your own layout from scratch either.
Nikola is neat, because it handles a ton of stuff for you. But depending on what you need, it may be too feature-heavy.
Sphinx is cool because a lot of Python projects use it for documentation, so if you get good at Sphinx, those skills will flow over into other projects too.
Lektor is really cool because it comes with a local content-management system/built-in server so you can write and organize your site in the browser.
You can also build your own with Python or shell scripts. I would advise against that, unless your needs are very simple or you don't mind sinking some time in, learning a bunch, but eventually getting further down the road and realizing that you might be better off just using a framework that already works and does what you want.
Check out StaticGen to see many of the static generators available. You can see what's out there and see which features you like or don't need.
Many static site generators support adding in data from static JSON/CSV/YAML/TOML/INI files, so the process of writing a script that fetches all of your data and stores it in that kind of file so that your site generator can use the data.
Anyways, that's a lot of stuff. Does any of this sound like what you're looking for or did I miss the mark?
Let me know if you want help with anything specifically, and I can try to be more specific :) Or message me on here!
First of all thanks a lot for the long answer and sorry for my late reply...
Real-time would be really nice but I absolutely don't want to have my keys in the front-end. A script or service in the backend would be great.
I already tried static site generators like Jekyll, Hugo, Pelican and Nikola. They are really great and I really like to use them.
A job that checks my Mastodon-feed or Blog and triggers a rebuild of Jekyll, Hugo or whatever would be pretty good.
Your idea with a JSON or YAML file sound interesting. I haven't thought about that.
So my problem is not the static site. It's more like how do I rebuild it after a blog post or a toot and what would be the most efficient way.
Thanks!
Cool! And now I totally get what you are wanting. So now we have a couple of options. You could go the pure python route and use the built-in schedule module. I’ve got a project called Fanbot that tweets a compliment at my brother every day that is similar to what you want to do. Or you could go pure Bash and do it through an hourly/daily cron job.
Either way, you’ll need a server to run this for you (or a serverless function). Options are an old computer you’ve got laying around, buy a raspberry pi for pretty cheap, or maybe a $5/month Digital Ocean box. AWS would also work, although I’ve found it to be a lot less user friendly. Netlify has functions that might actually handle the data fetching and optional rebuild for you. Not sure.
The recurring job would check each feed, and, if there had been any updates, download them, rebuild your site, and push the changes to your repo.
I’m happy to help you out with whatever combination you want to try. Do any of these ideas sound good? :)
Sorry I just thought of another idea. You might be able to scrape Twitter from the front end, bypassing the api and avoiding the rebuild step at all, but not sure.
I will have a look at your Fanbot project.
I just got it working with Hugo and gohugo.io/templates/data-templates.... You just have to disable caching and rebuild Hugo everytime.
Now I just need to use a cronjob or some other script to trigger the rebuild.
How do you want to do it with Twitter and the front end?
Something similar to this? Essentially fetch your Mastodon feed from the client side and dynamically show your tweets?
Personally I like the server side option better, but that’s just me. It’s nice because it gives you a handy backup of all your data in one place too!
This will just print the content of my last public toot from Mastodon.
I don't need any backup. I just want to show the last toot and link to it. So I think that would be simple enough.
When the site is complete I will check how long the hugo build will run on my uberspace.de host and then I can think about the frequency of building. Maybe every 30 min.
The nice thing about backup is you can check to see if there have been any tweets, and only build if that is the case. If you’re averse to deploying super frequently, maybe that’s a good idea. Otherwise, what you have sounds great! 😁
I am using Hugo to create my personal website. Hugo is an open source static site generator.
There are lots of themes created for Hugo. You may choose any one of them and tweak yourself if you need.
FYKI, I host my website in Github.