Reading scientific papers is hard.
I love reading – in an alternate life, I probably would have ended up as an editor of some stripe (I still think being editor for a science journal would be fun). But I am not fast at reading scientific manuscripts, and even now, I don’t always find it easy. Scientists are notoriously mediocre, dry writers – even if our professional evaluation hinges upon our ability to publish papers. Even when I’m excited about a paper, that enthusiasm quickly fades if I see the page count exceeds 15.
I’m teaching a class in Arctic Ecology this semester, and about half the sessions are student-led discussions of papers. I want the students to learn how to read a paper, connect it to the topics we’ve talked about in class, and build upon what was presented to explore new ideas. But, that means they have to first read and understand the articles. Which is no small task! I do try to pick interesting, reasonably long (i.e., short), at least somewhat accessible papers – that in and of itself often takes awhile. Even so, I wanted to provide students with some tools to understand how to break down a scientific journal article into manageable bits.
So, I put a call out to Science Twitter, and got some great suggestions! This post links to those suggestions, and a few additional thoughts of my own.
I supplied much of this to my class, and I’m happy to say our first discussion went great. They kept the conversation going for the full time allotted, everyone in the class spoke multiple times, and mostly they kept to higher level questions focused on synthesis and critique, rather than simply understanding. Because of the size of the class, we split into two groups, each with a pair of students to lead discussion. Grad students in the class helped keep track of who spoke and provide feedback on how the discussion went, while I switched between the two groups. Discussion leaders had worked together, and had to turn in discussion questions as part of their grade. Throughout the semester, students have to turn in five reading reports, during weeks that they are not leading discussion – this makes sure that at least some students have thought a bit more deeply about the readings and come prepared with a couple questions to discuss, too. We’ve only had one paper discussion, but I’m very impressed so far!
Anyway, twitter suggestions!
We started off in class with the 10 Stages of Reading a Scientific Paper. I try my best to be anti-elitist, and show that science is fun and rewarding, but hard. Even scientists struggle with the basics, sometimes. Sabriya Amira suggested this as a starter for that reason!
Christine Liu has a great template for breaking down an article, with six sections: Main points, methods, shortcomings, key figures, questions, and keywords. I gave this to students as an alternative for answering a series of questions for their reading reports – I really like this layout, and think it helps distill the main themes anyone should takeaway from a paper, at least to start.
Meredith Holgerson, faculty at St. Olaf College in MN, uses a similar “message box” format, with five points students should fill out: Hypotheses/Objectives, Main Issue, Methods, Results, and “So What”? This was adapted from SciComm resources, and I think is another more graphical way for students to try to interpret papers.
A couple folks then recommended several articles by Dr. Jennifer Raff, which I found useful. Her main approaches (somewhat adapted) follows:
1. Read the intro, which should frame the problem
2. Identify the Big Question
3. Identify the specific questions
4. Read the methods and identify the approach
5. Draw a diagram for experimental/research design
6. Summarize the results
7. Do the results answer each specific question?
8. Read the discussion/interpretation
9. Read the abstract – what do the authors say about the study?
I think this is largely a logical way to approach a paper, but I tend to do things a little differently. My main thing is that I don’t necessarily (or encourage students to) read everything in detail, first. The best items to focus on, after the intro, are all the figures. For paper discussions like in my Arctic Ecology class, we don’t need to drill down into the specific methods, often – I want students to focus on the big picture, and understanding the figures is more important for that. If I’m reviewing a paper, or if I were a science journalist, that emphasis could be very different, though!
Sarah Whorley of Daemen College also developed a Research Paper Reading Journal, where students summarized the article, provided the key figure, and identified some questions for discussion. Another very helpful template for an undergrad assignment on paper discussions!
Emily Bernhardt from Duke also shared some resources, posted to her website, including guidance on how to create a concept map. I shared concept mapping with my class, specifically, as a way to try and organize complex ideas that intersect in multiple ways. Her site also give helpful links on how to find literature and manage citations.
Finally, here’s a Science Magazine article that includes responses from a number of scientists on questions like “How do you approach reading a scientific paper?”, “What do you do when there’s something you don’t understand?”, and “Do you feel overwhelmed reading papers, and how do you deal with that?”.
Going through all these responses, two main things jumped out at me. Most importantly, there’s no one true way to read a scientific paper. It often takes trial and error to find a strategy that works for you. Some themes pop out – focus on the big ideas and identifying the objectives/hypotheses, look closely at the figures, think about what questions you still have – but no one had exactly the same approach.
I also noticed, however, that there was a consistent pattern in who responded to my twitter query and provided feedback. All the people who had developed a worksheet or resource of some sort (and were willing to share) were women. Admittedly, my twitter network skews towards women over men, but I did find it notable that the resources aimed at providing guidance on how to do a vital part of science were largely developed and shared by women. I don’t want to read to much into that, but it was certainly an interesting pattern!
If you have other resources that you think would be useful to add to this list, please let me know! I hope compiling them here will be a good resource for folks in our community, but this is (of course) by no means comprehensive.