Welp, I started this about 6 hours ago after the hackathon deadline completely sneaked up on me. In true hackathon fashion: the code ain't pretty, it was submitted an hour before the deadline, and by some miracle it works!
This project was inspired by a tweet I saw this afternoon:
tkasasagi🐻@tkasasagiSelfie 2 waifu.
Official Tensorflow implementation of U-GAT-IT: Unsupervised Generative Attentional Networks with Adaptive Layer-Instance Normalization for Image-to-Image Translation (ICLR 2020)
waifu.lofiu.com/index.html14:17 PM - 15 Apr 2020
As well as my friend and Twilio Evangelist Sam Agnew's response:
Sam Agnew@sagnewshreds@Ecmoy Someone should build a Twilio MMS phone number with this 🤔14:31 PM - 30 Apr 2020
This application allows you to send a selfie to a Twilio phone number and receive and "waifu" version of it in return that was generated using a trained model. The image conversion is all handled by the Selfie 2 Waifu project which was built by creke. I simply integrated Twilio and built the automation around it.
I could only really see this project falling under either the 'Interesting Integrations' or 'Exciting X-Factors' categories.
Since I waited until (literally) the last minute to build and submit this I didn't have enough time to record a proper video demo or anything but here's a sample interaction with this app from my phone:
I also encourage anyone to fork the repo and try it out themselves!
The code is MIT licensed and fully available on GitHub along with some (rudimentary) setup instructions.
Generate your waifu using Twilio MMS. Simply send a selfie to your Twilio number via MMS and receive your waifu in return.
This project was built on April 30, 2020 for the Twilio Hackathon on DEV.
To run this locally app locally you'll need to do the following (replacing the values as you go):
git clone https://github.com/Brodan/waifu-mms-bot.git npm install export TWILIO_ACCOUNT_SID='XXXXXXXXXXXXXXXXXXXXXXX' export TWILIO_AUTH_TOKEN='YYYYYYYYYYYYYYYYYY' export TWILIO_NUMBER='+15555555555' # run this in a in a seperate terminal # install instructions: https://ngrok.com/download ngrok http 5000 # copy your ngrok URL and export it # also make sure configure your Twilio number to point to *YOUR_NGROK_URL* export NGROK_URL=*YOUR_NGROK_URL* npm start
Once everything is configured and running, send a selfie via MMS to the Twilio number you configured and wait a few seconds…
I built this app using the following tools/technologies:
- Selfie 2 Waifu for handling the actual image processing/conversion
- Twilio MMS for sending and receiving images
- Express for the back-end
- ngrok for my localhost tunneling so I could get up and running quickly and not worry about deploying
- Puppeteer for headless browser automation
The biggest issue I ran into while building this app was figuring out how to automate the uploading of selfies to the Selfie 2 Waifu web app. I originally started out using an awesome tool called taiko that I usually use for these kinds of interactions. However, due to the page's file upload proccess I couldn't quite get automated uploads working. I then switched to Puppeteer which I had never used before and luckily it was just simple to work with and was able to accomplish what I needed.
The only other issue I ran into was working with asynchronous code throughout my Express server since I am a Node newbie and struggled with some of the
async/await syntax. This, along with the time constraints, is why the code is definitely not as clean as it could be and might mortify some of the JS experts.
In reality, all the credit should for this app should go to the Selfie 2 Waifu author, creke. Without their awesome app and the inspiration it caused I wouldn't have built this.
Additional thanks to DEV and Twilio for running this hackathon.