How to read papers without falling into a rabbit hole?

200

84

Often while reading a paper, I will see a citation that that seems more relevant than my current one. I'll switch to the new paper, only to experience a similar feeling. Or maybe I need to look something up on Wikipedia. An hour later, I'll still feel as if I didn't really digest anything. I will have skimmed 2-4 papers' introductions and conclusions, scratched some notes on the side, and generally feel overwhelmed.

Is this a bad habit? An abnormal feeling? How have others dealt with this to become more productive readers?

I'm fairly new to research, if that matters.

gwg

Posted 2016-05-12T20:32:05.970

Reputation: 2 099

44Nope - perfectly normal to begin with. Then you find yourself looking up the same paper from a different line of descent through references and you start feeling at home in the literature... In a new area this still happens to me 25+ years after my PhD – Jon Custer – 2016-05-12T20:57:50.727

18It's normal. A 'rabbit hole' is probably the wrong expression to use. By exposing yourself to more concepts and terms, your brain gains the tools required to quickly understand papers. If you ever wonder why experienced researchers seem to grasp a paper quickly, that's because they have been down many so called rabbit holes. Just be persistent, and have the patience to go down rabbit holes and enjoy the many ideas you'll encounter. They will pay in the long run. – Prof. Santa Claus – 2016-05-12T22:01:11.740

55Rabbit holes sometimes lead to wonderlands ;) – conjectures – 2016-05-13T10:53:09.820

5Rejoice in the overload. Often, when getting to grips with new concepts, I'll become completely overwhelmed by the influx of new information. At that point of feeling the sensation of being overwhelmed, it's easy to fool yourself that you've learned nothing. Take a break/sleep on it, and pick it up again later. Much more that you think went in! – spender – 2016-05-13T11:11:11.647

17Now if someone can answer how not to fall into the Stack Exchange rabbit hole! – Agustín Lado – 2016-05-13T18:58:33.660

3

I have answered this question here: http://math.stackexchange.com/a/617995/26580

– Superbest – 2016-05-13T23:03:45.333

3Relevant XKCD. – Andrew Grimm – 2016-05-14T02:17:03.633

1Tropes will ruin your life in a similar way. – Damian Yerrick – 2016-05-14T03:02:00.367

3Wait until you find out there is another species of rabbit who is digging a completely different hole on the same topic. – geometrikal – 2016-05-14T06:22:03.933

@Superbest, your answer is awesome. Just having someone with more experience say that BFS is an acceptable approach is great to know. I am finding that research takes a lot of intellectual courage, and knowing what the process feels like is useful. – gwg – 2016-05-14T12:17:10.937

@DamianYerrick I was planning on not mentioning "Wiki Walk" because it linked to that site. :O – Andrew Grimm – 2016-05-15T01:04:26.627

Interesting. This kinda reminds me of looking things up in a hardcover Oxford dictionary when I was learning this language back in my high school. Ah good ol'days! – Vim – 2016-05-15T11:00:35.487

pomodoro that shit, yo. – K. Alan Bates – 2016-05-17T15:45:59.660

@Vim ...this is how I learned English as well,and I'm a native speaker. I picked Return of the King from a shelf in my dad's library when I was 6 (The Eye of Sauron on the spine made me pick it up).He quickly snatched it away saying "No, no, no!You're definitely not ready to read that book!"He turned and put it back on the shelf then brought back FOTR and said"read this one first or that one won't make any sense"and then dropped 3 dictionaries on the side table.thud, thud, thud"You'll need those." I learned English from Tolkien. ...I naturally thought that everyone learned this way. – K. Alan Bates – 2016-05-17T16:59:48.147

@K.AlanBates I agree "ratholes" (or "rabbit holes"? Doesn't matter to me) in dictionaries are indeed a lot of fun. By the way you mentioned Tolkein.. I thought I had seen it said somewhere that this guy was very inclined to invent new words out of his wild imagination (which of course makes his works notoriously resistant to translation), so could you really make sense of all those words just via dictionary? Just curious. – Vim – 2016-05-17T17:13:44.160

@Vim ...well, Tolkien was a philologist who had a love for creating languages (Quenya and Sindarin being the ones most people would have heard of...I can't actually identify any of his others, but its understood that he did create other languages) so no: you cannot make sense of all of those words in a dictionary. It's what leads you to understand that you need to use the surrounding context to compartmentalize your understanding of things and "move forward" if you actually intend on finishing the books and not do endless, nonsensical research on fairly pointless things to your overall goal. – K. Alan Bates – 2016-05-18T07:19:37.653

@Vim so when trying to tackle Fellowship of the Ring as a 6 year old armed with a 6 year old's understanding of things and 3 dictionaries that don't contain all of the words one sees from Tolkien,you have to quickly adopt the mindset of "I don't care what that means" until you see it again, recall it and realize "this is probably important. Now,I can wonder: what is that?" I wanted to read the story,not sit with my nose in a dictionary.The dictionary was a means to an end given that my 6 yr old self did not have a sophisticated enough grasp of English to do it on my own – K. Alan Bates – 2016-05-18T07:31:10.887

Answers

131

Your problem is quite common among researchers. Actually it's not really a problem, being overwhelmed like this is just natural. Me and all my friends and colleagues face it.

How I overcame this issue: I try to focus on one paper at a time. Try starting reading the latest research paper on a particular subject and go back chronologically. Print the research paper in hard copy, leave your computer and cell phone (if possible) behind and start reading the paper. The point here is to avoid internet access. No matter what question you have, do not search it right away. Write it on the side notes and keep reading. Most of the time the answer will be in the later sections of the same article. Even if you do not get the answers, once you finish reading the paper, you can go online and find all the answers. That's how I read literature without falling in a rabbit hole.

Also, reading the Abstract first, then the conclusion, and then the rest of the paper is also effective.

For me, changing place of study also helps to focus and good instrumental music is always a plus.

Mirza Awais Ahmad

Posted 2016-05-12T20:32:05.970

Reputation: 1 346

1As a curiosity, what music do you listen to to help you concentrate? – Ant – 2016-05-12T23:01:26.027

4@Ant Classical music in general...Bach, Mozart, Beethoven, and Chopin in particular...are perfect...for me anyway. – Fixed Point – 2016-05-12T23:29:08.343

1

SCNR - my acoustic island of sanity: http://www.kdfc.com/

– Stephan Kolassa – 2016-05-13T07:13:52.310

1Until about 10 years ago I, too, would print out copies of everything I intended to read. Eventually, I noticed two things: certainly that the piles of paper were becoming unmanageable, but also that I hadn't got much use out of most of the printouts, beyond discovering other more-relevant documents. Thus, kinda a waste to print. – paul garrett – 2016-05-13T20:02:06.187

1@Ant, Games or Movies OSTs are good for me while coding or lab work, for reading, as Fixed Point said, soft music is perfect. you can find some playlists on youtube. my fav is Zack Hamsey though . . :) – Mirza Awais Ahmad – 2016-05-13T20:32:05.557

1@Ant for a moderner style, check some math-rock (yeah, there is a subgenre called like that) – Ander Biguri – 2016-05-13T23:40:18.903

I've read somewhere that music with lyrics (especially if it is your language) can alter memory. Also for my sake, since I am into classical music as well, it rather disturbs me than helping me concentrate. – Blue_Elephant – 2016-05-17T08:47:47.583

1Just like like baking. You don't start executing the recipe after the first sentence, you read it to the end first. – Agent_L – 2016-05-17T09:46:59.007

48

(For context, I am in mathematics, and worked on it for almost 20 years before the convenience of the internet...) As suggested by @JonCuster's comment, I think this is what is supposed to happen when one is studying (!). That is, I think genuine study of the literature is "going down the rabbit hole"... and/but not giving up or bailing out somehow. Sure, sometimes there's a "grass is greener on the other side" feeling, but when looking at the research literature it is entirely reasonable to be stubborn in the sense of insisting on explanations.

It struck me that you mention "after an hour, and looking at 2-4 papers"... when I often find myself looking at papers all day long with the conclusion being that there were things that I didn't understand, or wasn't aware of, or had forgotten, that have surprising/significant impacts on the issue I'm considering.

It is true that this (I claim genuine) seeming-inefficiency is wildly incompatible with the assembly-line notion of "research" that we find ourselves being pushed into for various reasons. Hmm... :)

Making notes with good bibliographic indicators, and typing-up these notes rather than leaving them handwritten, and dating everything clearly, is the only thing I know to keep back the deluge. Dating, especially, so that months or years later you can see at least your own personal chronology of awareness is very useful, I think, since it can explain things to your future self about your present and past self's behavior.

In a similar vein, rather than attempting to organize computer files "by concept", in some cases (when the concept is not entirely clear!) it is simply best to organize by chronology.

paul garrett

Posted 2016-05-12T20:32:05.970

Reputation: 44 682

20

I have previously written an answer to the math version of the same question. I see no reason why it can't be applied to this case as well.

The human tendency is to prefer worrying about the most recently raised concern: You are reading paper X. You see a term T you don't recognize. You decide to pause reading X to quickly look up T (of course, underestimating how long that will take). You find a paper Y explaining T. While reading Y you find another unclear term U. U has been encountered just now, but T and X were minutes or hours ago - of course U seems like the more urgent thing to figure out. Next thing you know, you're in a meeting about X, you don't know a damn thing about X, but you sure learned everything there is to know about U!

So the human mind, at least in my experience, operates as a stack (in fact a leaky stack, since our attention is quite limited and things at the bottom often become forgotten instead of just delayed). The stack happens to be the defining feature of depth-first search algorithms.

By many relevant metrics, depth-first search (DFS) is a particularly bad choice for reading papers. For instance, presumably the paper you are reading is expected to contain the most important information to you at the time; if there was one that seemed more important, you would read that instead. But with DFS, you will spend a lot of time reading other papers, only distantly related to the original paper - ie. you will waste your time on less useful things.

Because human knowledge is vast, your attention will become exhausted long before DFS hits the wall and starts returning to the original topic.

Probably the biggest reason more efficient approaches (for instance breadth-first search [BFS]) are not more common is that they require additional hardware. Namely, a pen and paper to write a list of things to look up as you read the paper, so that you don't have to split your attention between reading the paper and remembering this growing to-do list. One also has to fight one's own laziness when going to grab a pen and paper, and then (gasp) actually writing, which is much slower than thinking.

Also, the only important point isn't the order in which you look things up. There is also a pruning issue. Many people (including myself) will overestimate how important a term is to understanding the main point (quite a silly habit, since it's essentially trying to guess what it will take to understand a text you have yet neither read nor understood). With a BFS style approach, often it turns out that most of those things you thought you should look up don't really matter and you don't need to look them up. With the DFS, it is much harder to tell what terms actually matter, which ones are irrelevant and which ones become obvious by the end of the text.

Basically, you have to exercise your patience and rebuke your inner sloth. Finish reading your current thing first before looking everything up. Don't worry, you won't forget - just write down what needs to be looked up, with references to where it occurred if you are very worried. Yes, you do have to physically write things, which is clumsy (hence why I say rebuke your inner sloth) but absolutely necessary to overcome some crucial limitations of the human brain. You can type instead of writing, or highlight, or draw !'s on the margins - doesn't really matter, so long as you use a physical (as opposed to mental) means of recording it, and finish what you're reading before starting to read other things.

Superbest

Posted 2016-05-12T20:32:05.970

Reputation: 5 439

1Maybe not everyone knows the acronym: "DFS" is "depth-first-search", and "BFS" is "breadth-first-search". :) – paul garrett – 2016-05-14T17:43:30.877

1@paulgarrett Fair point - though I'd recommend reading the linked answer first (which does expand them). – Superbest – 2016-05-16T14:17:32.793

12

It is interesting that you used the concept of "rabbit hole" when you are new to research. Couple of things:

Uncertainty is Always There: You read it right. The "rabbit hole" is always there. In fact at first it is the "black hole" that sucks you in and you are amazed/sad/angry all at once!. This is normal because the area is new, you have no idea how to make something out of it. As you progress the hole starts to fill in more and more, however the uncertainty is always there but your approach at the research become more and more clear and becoming more smart and you will guess what your contribution will be. This bring me to the second point.

Contribution: Take your area, and work toward a contribution. You can never be 100% sure about a contribution until you could prove it in a respectable publication. This is the whole point, you dont fill the "hole" completely, however you fill it a little so the next researcher that comes to your area, feel a little more comfortable in the "rabbit hole".

Read Read Read!: Keep reading, take notes, see who is doing what, and then after a year or two, if I ask you about your area, you know the history and you know who did what. Most of the time it comes down to few people. You will be amazed how many few people are bringing your research area forward.

Dave Rose

Posted 2016-05-12T20:32:05.970

Reputation: 6 900

5Please re-read your third to the last sentence and fix it, it's confusing. – CGCampbell – 2016-05-13T17:49:58.517

5

Getting hold of a good literature survey paper helps especially if written recently...given that it exists for your topic.If so you probably wont do better than citing the relevant paper mentioned in the survey. Hand-books also help identify which papers ought to be the go to papers for citation.

akm

Posted 2016-05-12T20:32:05.970

Reputation: 51

4

One way would be to write down every term or phrase that spark your interest and look them up later.

The best way in this day and age, in my opinion, is to open up each paper you plan to read in a new tab in your browser and sort them in order of importance. For example, the paper that you are currently reading should be in the active tab; the next tab should be the paper you want to read after you stop reading your current paper, and so on. Then you can easily stop reading one paper, switch to another and deprioritize what you were just reading by moving it down the tabs list so you can get back to it right away or a bit later. You can also utilize favorites to add papers you want to go back to later.

However...

Falling down one of those rabbit holes led me to changing fields and a different career path. I am very happy about that and I think others might be too if they just follow their thirst for knowledge and do not resist it. It is very natural for a scientist (and for any person!) to be open minded and constantly search for something more interesting, fascinating, amazing, and readjust academic and personal goals accordingly. Whenever you find something interesting that seems more important than what you are doing now, check it out, maybe it is! If it's not, drop it like a rock and go to the next paper, or back to your previous one. You can get really, REALLY fast at multitasking like that using browser tabs/favorites to organize/reorganize your reading list. Use CTRL+F searches to quickly find the phrases you need. Modern PDF viewers also utilize tab systems, which gives you even more organizing power. Get good at this and those deep rabbit holes won't seem that deep anymore and you will find plenty of wonderlands down there with much less effort.

Arthur Tarasov

Posted 2016-05-12T20:32:05.970

Reputation: 4 092

2

As Mirza Awais Ahmad already said, if you know that there is one paper you need to focus on, a good solution is to print out the paper, turn your phone off, leave the office, and go read it in a cafe/park/at home/on holiday/in the library without the ability to download another paper. Bring a notebook with you and make notes on paper or in the margins. If there's a reference that seems important, just make a note of it for later, when you're back at your computer. Setting aside time like this can be a greatly enjoyable experience, especially if your normal mode is skimming.

If the issue is that you don't know which papers you need to focus on, an excellent solution is to join a reading group. Or, if there isn't an appropriate one in existence, start one yourself. Find some likeminded people with an interest in a similar area, decide collectively which paper to read next, and set a date to discuss it. This not only provides a great motivation for sticking to the target paper, but you also get the help of other people in deciding what's important, and the subsequent discussion can be an enormous help in understanding both technical details and the broader context. You'll almost certainly be doing your colleagues a favour as well by starting a reading group, and it will also give you experience organising groups of people, which is invaluable for a future research career.

Nathaniel

Posted 2016-05-12T20:32:05.970

Reputation: 1 168