Meta: how to read this post?
I’d recommend just a top-to-bottom read while thinking about what you agree with and what you don’t. But why shouldn’t we just follow this approach while reading a research paper? Because this post is written for the sole purpose of someone reading it in this fashion – while research papers are not!
Motivation
As a young researcher I do read a lot of research papers as part of my job. Last year (2023) I read 250+ papers. This took around a thousand hours. And it was slowly in these “paper reading” sessions that I realized that I am not reading papers the right way. I then tinkered around a lot to figure out a way that works for me. I hope it helps someone else too.
Research papers are not read for the same reasons they’re written for.
Why to read research papers in the first place?
This question has a surprisingly difficult answer IMO, and once you clear up in your head why you want to read a paper, this leads to the “how” part pretty naturally. Here are the reasons why I personally decide to read papers:
- I somewhat like it. (whether you should do something because you like it is another big question, but for this one I’m positive because it is meaningful too).
- It helps me in moving towards my long-term goal research goal of solving XYZ (abstracting it without loss of generality). Why does that particular long-term goal excite me? That’s for another post, but this is how reading papers helps me in this goal:
- Firstly, it tells me what other people are interested in similar problems as me and what kind of work they are doing.
- To better understand the field and figure out what open problems still remain unanswered.
- To build my research taste. (which is a highly subjective and uninterpretable thing but IMO pretty important to develop). Update: I recently came across this cool short guide by Chris Olah on some exercises to build research taste. In a way, this is an addition to those exercises.
- [IMP.] To help me become better at problem solving and research intuition. This is extremely important.
That’s enough to motivate me. By the way, this is in no way putting papers as “better” than blogs or lectures. Many a time they are much better for learning. Similar ideas however do apply to those things as well. Before we think about reading papers, let’s look at the why and how they are written and structured. And it’s important because papers are not written for the same reasons why they are read.
The Anatomy of a Paper
A paper is an attempt to further a research field by sharing something that was not already known. It could be about a new problem, an experimental set-up, results of experiments, a theorem (with proof or empirical justification), a hypothesis, a refutation of prior works, etc. All of these things seem very important and different kinds of people like different kinds of things here.
A research paper has (broadly) the following parts:
- Abstract: This seems to be the most important thing for someone who just wants a TL;DR. A one-para summary of the whole work.
- Introduction: The set-up of the problem and explaining in detail the gap in science that the paper is trying to fill.
- Related works: Prior works done in similar areas and how this one is different.
- Idea: The main idea of the paper. Usually never called “idea”.
- Experimental setup: self-exp.
- Results: Empirical results to show that the idea is justified.
- Ending: Split into conclusion (another one-para summary), scope (conditions under which the idea works), discussion (hypotheses?), future works (if someone is interested in building on it).
- References: What prior papers does it use/build upon?
- Appendices: What is not important enough for a 6-page paper but still something they did and could be useful to others.
So, how should we read it?
A disclaimer first: this is highly opinionated. It just might be that this is what works for me. In that case, I hope it helps you think about this as well. For comparison, let’s say reading the whole paper from end-to-end (excluding references and appendices) takes 2 hours. Once you start reading a number of neighboring papers, some papers become a pretty quick read (maybe 30 mins), but let’s start with 2 hours for now. Instead of doing this top-to-bottom thing, here’s what I like to do:
[2 min]Skip the abstract: The only way I think this helps me is to decide whether to read the paper in the first place. Otherwise I really don’t like them; they’re like looking at the scorecard of a cricket match: good for entertainment purposes but useless for an upcoming junior cricketer.[~30 min]Fully, deeply understand the introduction: This is the golden part where I spend a lot of careful time. Ideally the goal is to fit every small detail and nuance of the problem in my conscious memory and motivate my subconscious to think about it by attaching some emotion to the gravity/importance of the problem.[~40 min]Take a walk, detach yourself from reading, and think! I take a break at this point and try to think about this: “what would I have done to try solve this problem?”. This is an exercise motivated by the saying “learn from other’s mistakes (problems) because you’re not gonna have the opportunity to make (face) all of them yourself.” Come back to the paper after a break.[~30 min]Read the idea section: Finally read about how they solved the problem. Meanwhile compare with my ideas and see what all I (or they?) missed and why. Here I’d make a quick peek at the main results (usually a single table or chart) to see if their work makes sense.[5 min]Skip experimental setup and results: Unless I want to work on a very similar problem, this feels like a waste of time.[12 min]Read discussion and future work; skip conclusion.: same vein as the idea section above.[1 min]Skip references and appendices (unless something seems interesting).
These [X min] things are almost never followed and different things mix into each other all the time, but the main point I’m trying to make here is that what you can learn from a paper is much more than what is written in the paper. I also keep a list of “notes”, which is a short handy list of pointers/takeaways that I either want to think about or discuss or keep for future reference.
Doing this takes me roughly the same amount of time (2 hours) as a top-to-bottom read would, but I feel that I learn much more for the long-term. Do you have your own way of reading papers? I’d like to know, please comment! Did you try this? Let me know how it went! Thanks. (and sorry for amorphous posts; that is an intentional goal for this year).