Reading Research Papers - PART I

How to master a new body of literature

Let’s say there’s an area you want to become good at like speech recognition

or Text summarization or building a chat-bot.

Firstly,

1. compile a list of papers (from arxiv +medium, blog posts)

2. skip around the list that was compiled.

Lets say we want to master the domain of speech recognition systems.

We start of with 5 papers

• we read 10% of all papers

• quickly skim through and understand these papers

• based on that let’s say we decide, paper number 2 is dud and it doesn’t make sense

• further we decide paper 3 is a really seminal one, so we put a lot of time to read and understand that.

• Keep adding the papers that are cited to the reading list in order to understand the best paper from them.

• read the set of papers

Rough guidelines on the number of papers to read.

If we read 5-20 papers

• we have a basic understanding of an area.

• be able to do some work, apply algorithms .

• not enough to do research or be at the cutting edge .

  
  

If we read 50-100 papers in an area

• gives us a solid understanding of an area.

• helpful to do cutting edge research.

How to read any Research paper:

A bad way to read a paper is to go from the first word to the last word.

Take multiple passes at the paper.

Here’s how Andrew Approaches to read a paper.

1. Read title/abstract/figures (first pass)

2. Intro + Conclusion + figures + skim the rest (skip related works)

3. Read the paper but skip the math.

4. Read the whole thing but skip parts that don’t make sense

Some questions to keep in mind.

1. What did the authors try to accomplish?

2. What were the key elements of the approach?

3. What can you use yourself ?

4. What other references do you want to follow ?

Sources of New papers?

• Twitter
• ML subreddit
• Follow NIPS/ICML/ICLR/ACL/EMNLP conferences.
• Arxiv sanity

To understand the papers deeply .

• To understand the Math

  • Read through and take detailed notes

  • Re derive it from scratch

• To understand the code.

  • run open source codes

  • Re-implement from scratch

Advice to navigate a career in machine learning - PART II

There are so many options, to work on exciting things in the field of ML/DL.

Assuming that most of us want to do two things.

Goal.

• a job (big company or startup)
• a PhD

Mostly Goal is to bag ( to do important work )

1. How to get a position

What do recruiters look for (recruiters/ applies for PhD admissions too):

• skills ( on ML, quiz on ML skills and coding ability)

• meaningful work (know how to make the stuff work) in a meaningful setting [ IMP ]

Failure modes to avoid.

Avoid following only broad approach or only depth approach . (only breadth or only depth)

Backing up a broad approach with tiny projects reflects a lack of in depth understanding to recruiters.

Much worse is to have no working knowledge about other domain

and a thorough depth in only one.

A Better way is to cover the horizontal aspect by building the foundational skills (course work+reading group).

Depth aspect is achieved by doing relevant projects, internships, research work.

Some pointers to keep in mind.

• Focus on being jack of all trades but master of one area.

• Strong candidates for jobs are: T shaped individuals.

• Recruiters are not impressed by volume.

2. Selecting a position

Aim working with great people / projects.

• Focus on the particular team you will interact with (10-30 people).

• The manager has a good influence on what projects one works on.

• Focus not on brand of company.

Working in a company:

In a giant company with 50,000 people, let’s say they have 300 person AI team.

• If we get a job offer to join the 300 person AI team that would be pretty good.

• Even better would be to get a job offer to join a 30 person AI team.

• So we know who our manager, peers are.

Mistakes to avoid.

A real life incident.

A student got a job offer from one of the giant companies, that has a great AI group.

His offer wasn’t to go to the AI group, his offer was to join them and then he would be assigned a team later.

When he joined, he was assigned to a back end payments team, and he didn’t really like that work because his aim was to work in the AI team.

Be careful about the marketing of rotation programs as well.

Some times rotation programs sound well on paper, however we need to be careful what job offer we get in the end.

Be sure of all the specific nitty gritty details before joining any team.

There are some companies, which are not glamorous and do not have cool brands,

however they might have an elite team of 100 people doing great work in machine learning.

Future leads:

Andrew’s lecture ends by him pointing towards the next phase for the evolution of machine learning, which is for traditional industries that do not have shiny tech things, because the value creation there is much bigger for example in the agriculture, healthcare, manufacturing industries. (low hanging fruits)

Long term tips:

• Steady reading, not short term bursts. ( Take leverage of compounding )

• If early in career:

  • Work on things that help learn the most.

  • Do important and meaningful work

References.

 CS230 Stanford