Managing research ideas

I received this email today:

Dear Professor Hyndman, I was wondering if you could maybe give me some advice on how to organize your research process. I am able to search the literature on a certain topic and identify where there is a question to work with. My main difficult is to organize my paper annotations in order to help me to guide my research process, i.e, how to manage the information gathered in those papers to compose and structure a document which can represent the research developed so far. I have been looking at different tools such scrivener, Qiqqa, papers2, etc but I am not sure if one of these tools would be the right way to go. To be honest I am not even sure a tool would do what I am looking for, not just organize references and annotate pdfs but to get more control of my research process. I appreciate if I could get your thoughts on this subject.

Here is my reply:

That’s an interesting question, and probably the answer depends on personality and style of working. I can tell you what I do, but I wouldn’t like to suggest that everyone should work like me.

I always start with a problem that needs resolving, and I think about how to do that. I never consciously start with other people’s research. I keep an electronic notebook of research ideas (on Evernote) that I might work on some day. Sometimes these ideas occur to me while reading other people’s work, or when listening to a talk. But they often arise from consulting projects, or discussions with people who work in a commercial environment. I write a few ideas down, and I leave them in my notebook, adding to them whenever something new occurs to me. Occasionally, if I’m bored in a taxi, or I’m waiting for something, I’ll pull out my notebook and flick through the pages to see what’s there.

After an idea has been in my notebook for a while (anywhere between a few days and several years), I might feel ready to tackle it. Normally I just start coding in R or writing something in LaTeX. Occasionally, I’ll check a paper and have a look at what other people have said. But mostly I work independently.

At some point during this process, I start collecting papers that might be relevant. All my papers are kept in Mendeley, but I don’t really use annotations. For a particular project, I create a group in Mendeley and start adding papers that I think I will need to cite.

Much later, when I’ve written up the main ideas, I go back to the group of research papers and think about how they fit in with my own work. Then I write a literature review which places my work in context. But it is important, I think, to understand that the literature review tends to be written after the main research ideas have been written up. I don’t find it helpful to read lots of papers before developing some research ideas. Of course, because I have read a lot of papers for other purposes, I have absorbed a lot of ideas sub-consciously, so my research is not as independent as this seems. But when I’m doing it, I am almost never thinking of someone else’s work.

I wonder if any readers have alternative approaches that they would like to share?