DEV Community

Discussion on: From Rails to Elixir: Know Your App

Collapse
 
trinitytakei profile image
Trinity Takei

Hi João,

Great post! I'm REALLY looking forward to these figures (if I understand correctly you'll let us know in the end):

Rails: x hours development time, y $/month spent to monitor z pages [reasonably fine-tuned, eg after the jemalloc tweak mentioned above], w resources (time, money, ...) to keep it going
Elixir: p hours development time, q $/month spent to monitor r pages, s resources to keep it going

I'm working with Ruby/Rails since 2005, enjoying Elixir a LOT (it's the second time I feel like this in my life - the first one was when I found Ruby!), interested in scraping (have been doing it for the past 15 years, professionally, academically, OS and every other way) thinking about (partially) switching to Elixir). Oh yeah, and I feel similar about Go (not debating it's power, but not really excited about the implementation details - it's C-like, for starters, which is a dealbreaker for someone thinking the Elixir way is the best stuff since sliced bread, err, Ruby.)

What I'm secretly hoping is that someone can prove me that for this setup, Elixir/Phoenix is a better choice so much so that I can justify spending time on rewriting my Ruby tooling from scratch -considering that my time is really scarce (If I had plenty of time, I would do it out of joy, even if I knew the answer - however, right now I have 2 jobs, 2 kids, 200 other things to tend to etc).

The thing is that as someone currently working on a Rails codebase that's monitoring 1000s of sites around the clock, I know that the shiny stuff that comes with Elixir will get you only so far, and eventually you are going to fight language-agnostic issues anyway (how to detect which of your scrapes are broken, how to maintain/fix the issues the quickest way possible, how to stay under the radar if you are grabbing massive amounts of data (hint: concurrent hammering, even is relatively cheap, is not it), how to deal with rogue data... and more).

Anyways, this comment is way too long already, I'm wondering if you'd like to move the conversation to email - I hate communicating via web forms ;)

Collapse
 
jdcosta profile image
JD Costa • Edited

Hi Trinity, we are on the same boat then!

it's C-like, for starters, which is a dealbreaker for someone thinking the Elixir way is the best stuff since sliced bread, err, Ruby.

Spot on.

What I'm secretly hoping is that someone can prove me that for this setup (...) considering that my time is really scarce

Well, I hope that you can find some value from what I'm planning to share in future articles - an incremental migration to Elixir by having both systems working together during the transition phase. You can do a partial rewrite of your system by migrating something that you feel would fit Elixir best in a first stage, just to experiment with it and see if you are getting value from it. In my case I'm planning to migrate the Page Downloader first, keeping all the other components in Rails' land, communicating via Redis (using the Sidekiq protocol).

you are going to fight language-agnostic issues anyway

Absolutely. Using Elixir may offer you better control and tooling but the typical scraping/crawling problems won't go away with Ruby.

I'm wondering if you'd like to move the conversation to email

Sure, it sounds like our circumstances are very similar, it would be great to share experiences/thoughts with someone with the same set of challenges in-hands. I'm now following you on dev.to (here) and Twitter, we can get in touch through either and share email addrs.

Thanks for the feedback!