I know there are a bunch of .NET twitter bots out there, there are even packages such as TweetInvi that can help me pretty much setup a bot straight away. However, I’ve decided to take the hard road on this challenge and do it myself. I'd like to get more in-depth knowledge about what it takes to create a bot, to deploy it and explore possible scenarios that can arise after deployment as a result of the choices made during development.
This three-part series contains a project that will take a look at creating a .NET app which will listen to specific tags I’ve pre-set and will retweet those tags. After that, we will then go on to creating endpoints for influencing what the stream should and shouldn’t listen to, hitting the “rules” end point for twitter. We will also look at deploying this .NET bot onto my newly purchased Raspberry Pi (the last two objectives will be in no particular order).
Part 2 : The rules
Part 3 : The Pi in the sky
Starting with a console app to test
I wanted to get to grips with how twitter outputs its stream. The most straight forward way for me to get into it was to create a simple console app that has a HttpClient, which consumes whatever comes from the never-ending stream; and that’s what I did! Also by the way, I made a call (outside of the app using postman) to the twitter stream rules endpoint. This is in order to create a rule for my filtered stream to listen to, when providing me with messages (link here). This is just to get things started until I get to the next episode in the series.
What happened in that pic, was it WebSockets?
It was the use of HTTP streaming, a method whereby the response to the client is held open by the server with basically no end in sight until the server decides so. If you would like to find out the differences between these two WebSockets and HTTP streaming, check this brief explanation in this blog here and why I chose to go with this rather than WebSockets.
To sum up that short blog, I’ve taken a look at the twitter Api documentation which states that it delivers “...Tweet objects in JSON format through a persistent HTTP Streaming connection.”. This is what essentially made me choose HTTP Streaming instead for this occasion.
We got it working, time to do something with it
After cleaning it up a little, and established being able to create a “listener” to these tweets, we will look creating the action of posting a tweet whenever something pops up.
This new code is the shortened version of the first example, we just separated the responsibilities and placed them in different classes. The TwitterStreamQueries class (as the name suggests) is responsible for the establishment streams, whether it’s the filtered one (as shown) or any other that could be applied in other scenarios. It takes in a TwitterClient class that is responsible for instantiating things such as client credentials and base Uri etc.
The GetFilteredStream method is probably the most interesting thing here because we pass in a function as its argument. This is to provide the filtered stream the open-closed opportunity. This in return gives us the flexibility to chop and change the function argument however we like; without much, if any, changes to the GetFilteredStream method. To start with, I created a function that acknowledges a message for iteration's sake, surprise... it works 👍.
Time to create a process that will retweet the message that comes in.
The retweeting process
As the title suggests, I now dive into what it takes to be able to create a successful retweet request. One thing I do know is that I cannot use the same client, as twitter has a different authorization header requirement for the post request (OAuth1.0 instead of OAuth2.0 which uses bearer tokens).
There were many ways in which I could have ensured that the client used for a particular process utilizes the correct OAuth header for its intended purpose. An example of this could be creating a separate client for each OAuth process (I.e. OAuth1 client and OAuth2 client). However, that is too much for a single purpose-built app such as this.
For this particular case, I thought to go with assigning the header required for each case in their own respective method. It’s a simple enough solution that only cares about listening to the stream and retweeting, I won’t faff about with it I reckon 🤷♀️ (being a lazy dev here – don't copy me!).
Speaking on the request process, I particularly struggled with OAuth 1.0, holy moly 😭. At first, the idea of having this crazy looking value that required the existence of a bunch of keys and secrets seemed daunting. I was later able to come to terms with the fact that the only real work being done is just generating the OAuth signature. This page in twitter and this blog helped me figure out the requirements needed to generate the signature for the request.
The structure of the whole app
I quite enjoyed separating the process from the library of code containing the logic. What I mean by that is the console app that is essentially running this process doesn’t actually have any other physical files inside it that help it run the twitter stream or retweet etc. That lives in a different project whose output type is a library.
The structure of “listen and retweet” process
This is just to give a clearer explanation to what was mentioned above about the GetFilteredStream methods function as an argument argument. The client just listens to the stream, if anything pops up it invokes an action that it knows not much (if anything) about, as mentioned before. In this case, we're invoking the retweeting process.
Clean up and putting things on a background service
To clean up, I decided to make the console app into something more suitable. I’ve introduced a class that inherits from the BackgroundService class, named TwitterStreamService. The BackgroundService class is a “base class for implementing a long running hosted service”, I couldn't have said it better myself; (link here).
I used to inherit from IHostedService myself but I had to deal with implementing its other methods such as start and stop async. This BackgroundService class does so and only lets us deal with one method, it takes care of the rest. You can see how it's set up in this (repo here).
In order to start making this background service work, I would have to do a couple of things first in the program.cs file; The first thing would be to create a new instance of the HostBuilder (via the Host.CreateDefaultBuilder method), this would help in establishing a ground for me to create this console-based service and add any dependencies needed to it. The second step is to actually register those dependencies, some of which can be seen in the constructors of some of the pictures already displayed.
I’ve taken a note to mention the significance in ensuring that the twitter client stays as a transient, as I wouldn't want to mix up OAuth requirements between the streaming (which wants OAuth 2.0) and the retweet post requests (which wants OAuth 1.0). The twitter stream and the commands which use these transient clients can be singletons themselves.
I’ve probably made an unnecessary meal of this, but I’ve learnt quite a few things! Namely, about HTTP streaming and how certain values in the response headers can hint at the intent of the response itself. It was also fun adding background services and revisiting hosted services in console apps. The OAuth 1.0 process was probably my biggest struggle, coming to terms with something that looks fairly daunting at the start is an achievement within itself; and it provided me with new knowledge in OAuth 1.0 which is cool!
This link here is the git repo to this project, let me know how many wtf per seconds you experience 😂
My next step for this will be two things, to establish some CRUD rules operations via API endpoints (using the latest .NET 6 features – which should be fun) and to also attempt hosting this in my raspberry pi (in no particular order). Can’t wait, see you there! 💨💨💨🕺
Top comments (2)
This is really nice, thanks for sharing. BTW, Twitter currently has an OAuth 2.0 beta API for user authentication, so hopefully things will get easier on auth in the future. Neat that you're running it on a Raspberry Pi, too!
Ah thanks! I hope so too so it makes the list of required things to initiate requests much simpler. Thanks, I haven't finished it yet but I'm on the way 💪