DEV Community: Ane Berasategi

What I learned during my Masters in NLP

Ane Berasategi — Tue, 26 May 2020 06:34:53 +0000

I had the idea to write this post by the Github graduation initiative.

I started my Masters degree in NLP in LMU Munich, Germany in 2018. I had been interested in machine learning for about a year and had completed Coursera's Deep Learning specialization. I wanted to learn NLP more deeply and become a kind of expert in the topic. That's what a Masters degree is supposed to make you, right? After spending 2 years receiving lectures on it and doing programming assignments, I should be able to contribute to the community. Or so I thought.

This degree was built for 2 types of backgrounds: computer science and linguistics. There were some introductory classes for both, which means you wouldn't need one half of them. Because the students had such diverse backgrounds, the bar was set low.

Go beyond lectures

I realized if I just went to class and did the assignments, I wouldn't learn much. So first I looked for more advanced courses. I took the most advanced computer science courses I could find. These were very demanding but there was much room to grow.

You probably won't have much time left with student jobs or assignments, but if this situation applies to you, I strongly encourage you to use the resources provided by the university in any way possible. Attend other lectures as a guest, read the course literature, ask questions in class and try to talk to the professor or instructor about the topic, they will tell you things they don't mention in class.

Read papers

In NLP, the field has been moving incredibly fast in the past few years. The course materials we were given were trying to keep up but they weren't. Using Python in assignments was very recent, and there was barely one Deep Learning course in the last semester. I realized if I really wanted to learn what was going on in NLP, I would have to learn it on my own. I started reading papers.

I got a paper that seemed introductory, I didn't understand much of it so I went to the papers that this one was based on. I didn't understand these either so I kept going back and accumulating papers until I had a 2003 paper I could understand, and 15 others in my queue.

I slowly started to go forward in time and understand the basics of NLP, in a way that I wouldn't do during my lectures. I used this opportunity to make a Github repo with my notes, which has been growing steadily since then.

Understanding papers was challenging at first, but with practice you get better and can understand better what they present. In a way, I think this arduous and long task of reading so many papers was my actual Masters in NLP.

Do programming projects with a professor if you can

The programming side in many of my lectures were optional, which meant most people used their energy on the other demanding mandatory assignments. But I saw an opportunity here. I could do a project, have much of the professor's attention because they wouldn't be busy with other students, and have a more personal mentoring.

And that's what I did. My professor/mentor was a doctor in Linguistics turned NLP researcher, who offered a very fresh view on NLP, since most of the other professors were computer scientist, and their ideas came from the other side.

This professor didn't challenge me on the coding side, but she made me difficult questions about my results, their interpretability, meaning and impact, and forced me to think from a completely different perspective than the one I was used to.

Many times I'd spend hours coding and finally get to have some results, and I'd think 'ok work done'. But then I'd show her my work and she would ask me a thousand questions and I'd have to go back and think. Not think about how to code this or that, but think about what the results were telling me. It was frustrating and fantastic, I learnt so much.

Unfortunately the semester ended and I was faced with more workload, but there was so much material to work with and I want to go back to it. This is the project I did: exploring the semantic similarity between contextualized embeddings.

Own your master thesis

By the time I had to start thinking on the master thesis, I had read many papers, knew what the field was currently doing and I had some sketches of research ideas I could work on during my thesis. I looked for approachable professors who'd want to tutor students for a thesis, and I did a tour of professors, seeing what projects they were working on.

I was surprised to see that they were delighted by my initiative, that I had taken the time to read papers and even to have my own research ideas. They told me most students just take any available project, follow the instructions of the professor and just present something.

I realized then how a little initiative can make you stand out in the crowd. In the end I picked a project suggested by my supervisor, but I made sure to make it mine. I tried to really deeply understand it, read the papers related to it, which by this time it didn't feel like much work.

In the beginning I didn't understand what I was doing, I was just doing what he told me. But slowly, by asking question and realizing my knowledge gaps, I started to own the thesis. I started to talk more in meetings, come up with ideas, and to just be in control of my own thesis.

As of May 2020, I'm still working on it. I recently managed to replicate a paper from April and even improve its results by 2%. I don't know what will come of my thesis or if we'll manage to publish a paper, but I wish I could work on this longer. I'm learning so much.

Conclusion

In the end, whatever degree you choose, you will only learn what you want to learn. So I encourage you to put in the work, do your best. There will be many people graduating next to you, so do the work to stand out.

2019 in books

Ane Berasategi — Fri, 27 Dec 2019 20:15:02 +0000

I've always had a book on my nightstand but when I started uni and bought a smartphone I started reading less and less until in early 2017 I decided to read seriously again.

It's been 3 years since I started and I never believed I would read so much and enjoy it as I do. I set some goals for this year and I'm proud to say I reached almost all of them.

These were my goals for 2019:

Read 20k pages
- After having read 17k in 2018, I thought 20k was my limit and it'd be nice to go all in and try to reach it.
Read 10 books in Basque
- It's my mother tongue and unfortunately I've somewhat abandoned Basque literature.
Read 15 nonfiction books
- Because if I'm going to get serious about reading and read many books, I might as well learn something in the meantime.
Read 10 classics
- I noticed I had barely read any classics so I hoped to fix that.

And what I actually achieved:

I read 24k pages
- This is the official number, but this year I quit some books I just couldn't go through so the real number is closer to 22k.
I read 4 books in Basque
- I tried very hard but either the books I chose were all boring or I'm out of habit at reading Basque, but I'm not giving up.
I read 29 nonfiction books
- Big success with this one, I read many great books and learnt a lot.
I read 10 classics
- I didn't enjoy them very much but I did it.

I got 3/4 of my goals, read many fantastic books and my to-read shelf in Goodreads is longer than ever. I would say it's been a good year!

These are my top 5 books in 2019, in no particular order:

The Physician, Noah Gordon

I was a big Medieval nerd as a teenager and The Pillars of the Earth and The Physician were all-time favorites, I wanted to read it again and it made it into the top 5 despite it being a re-read. Unfortunately I didn't enjoy the sequel as much.

The neuroscientist who lost her mind, Barbara K. Lipska, Elaine McArdle

Not a magnificent nonfiction book but it got me interested in neuroscience and mental/brain illnesses. It also freaked me out. I read more books about neuroscience after this and it debunked many myths I had about psychosis, schizophrenia, bipolar disorder, etc. Somehow I thought I knew what they were but turns out I had no clue. I look forward to learning more in 2020.

Digital minimalism, Cal Newport

Each book from Newport has been in my top 5 in previous years, this time it was Digital minimalism. His books are so inspiring, and make so much sense, they make me want to improve so much. I think about his ideas even months after having read the book. I can't recommend his books enough.

Into thin air, Jon Krakauer and Endurance, Alfred Lansing

Small cheating in this one, both books are about endurance, one about a failed Everest expedition and the other about Shackleton's incredible voyage to Antarctica in 1914. These books grab you and don't let you go, I had nightmares with them and I felt the despair they felt in Antarctica and the cold and desperation from the Everest expedition. Extraordinary stories and very well written.

Midnight in Chernobyl, Adam Higginbotham

After watching the HBO series I felt some shy pride when I understood the very basic concepts of nuclear energy and I decided it was a fantastic opportunity to read about the Chernobyl nuclear plant and the 1986 disaster. It was terrifying, complex terminology sometimes but worthwhile.

Summary of 2019

I'm proud I read a lot but even prouder that I managed to be regular and read at least 1k pages a month. Sometimes life gets in the way or you get stuck in a book and lose motivation, but if you have a habit of reading, you keep reading and reading until you get out of that block. Picking up a light fiction or young adult novel, or a book whose topic you're interested in are some of the tricks that get me into enjoying reading again.

Ane

@aberasategi

Instead of reaching for goals, focus on developing habits - @farnamstreet

17:04 PM - 14 Jan 2018

7 30

Goals for 2020

2019 was a hardcore year reading wise, I don't think I can read any more intensely without compromising other areas of my life. I definitely want to invest more time and effort into reading in Basque, so my goals for 2020 are as follows:

Read 20k pages
Read 5 books in Basque or 2k pages
- I like the number of pages goal, books can have different lenghts and I just picked up a biography that's at least 800 pages, it seems to involve complex vocabulary and if I manage to read it, I'll consider it a huge success.
Read 25 nonfiction books

On to the next decade!