The start of a journey

Applying to graduate school was stressful and it required a lot of careful thinking about potential advisors, research interests and places to live among many things. I recently was offered a position as a PhD student in two very good universities with researchers at the top of their fields. I was extremely excited and euphoric both times I read the admission offer emails. I think that like many other people who have tended to be on the far end in terms of intellectual curiosity (aka nerdy-ness) since a young age, entering a world-class institution to be trained for high level research is a dream come true. And on top of that you get paid to do it. Not very much but still, you can focus 100% of your time on doing something you’re passionate about!

Reflecting on my own experiences, what I know, what I want to know and especially what I hope to be researching as a PhD student has been on my mind lately. It certainly can get terrifying at times. Regardless of the fact that I have been found to be worthy of a position as a PhD student and that I came off as qualified, it doesn’t always feel like that. It’s often a mixed feeling of “I can do this!” but also “Can I really?”. So being the person that I am, I came up with a loose plan for the next few months before I embark on my PhD journey. The idea is to work on skills that from my (naive) perspective I think will be useful as a researcher.

What is research to me?

The way I see it, good research is often about finding a mapping from solutions of fully or partially solved problems to new/unsolved problems. In the context of computational cognitive science, this can be using techniques developed in a totally different context and adapting them to the human intelligence/cognition problem. For example, during my undergraduate, I was interested in a specific cognitive phenomenon called categorical perception and I used a deep neural network to model functionally what might be happening in learning systems that have some of the properties of neural networks like the fact that they learn distributed representations. Deep neural networks were not developed as models of specific phenomena in brains or minds (although loosely inspired by them), they were developed as a powerful statistical learning tool that set a new paradigm in artificial intelligence research. I don’t really believe in the 100% novel ideas. I just don’t think human intelligence works that way. Even totally groundbreaking ideas have some grounding in other ideas. There’s a reason we often use the metaphor “standing on the shoulders of giants”. We spin up new things by reusing old things we know in novel ways. The novelty of an idea is probably a function of the distance between the ideas that the analogies and mappings used to come up with a solution connect. If this is true, then doing research is in part feeding your intellectual curiosity, letting it ride wild and asking a lot of questions. A lot of what you do will inevitably fail but that’s just part of doing science.

How should I feed my intellectual curiosity?

The space of answers to this question is like vast. Nonetheless, there are some constraints I can think of that would be helpful. It can be hard to not go in all directions and end up feeling like you’re still in square zero. For an aspiring computational cognitive scientist, getting into the habit of reading research papers is one way of feeding your intellectual curiosity. It’s hard to know what to read at times but I like to build my way up from the ground by looking for fundamental problems in the field. There is also a wealth of resources out there put together by people who have already started this journey (some of which I’ll link below). Just pick one and stick with it! I think one of the more important things is to engage deeply with what you’re reading: summarize it, be critical, compare it to other things you’ve read, ask questions about the solutions presented, etc. Here are some of the questions I’ve come up with to ask about any research paper:

  1. What is the contribution of this paper and the solution presented?
  2. What are some of the problems with this solution?
  3. Is this solution unique?
  4. How would I improve this solution?
  5. Is the design of the study valid? Does it really measure what it intends to measure?

Another way of feeding your intellectual curiosity is finding textbooks, web-books or just books pertaining to your field of research that you can go through end-to-end or select chapters from to go through. The advantage with those is that someone has already thought about what they think is important for the specific topic they wrote a book about and it can be an invaluable resource to point you towards other relevant literature to read as well. Not only that, they often contain questions that make you think about what you’ve read or implement/write up solutions to problems.

For computational cognitive science, being deeply familiar with technical tools is definitely important. I like to implement algorithms from scratch using a minimal amount of libraries in Python. I want to emphasize the “from scratch” aspect because it’s too easy these days to just copy/paste code from the Internet without understanding what it’s doing! I’ve found over the years that implementing an algorithm is one of the best ways to understand how a model works. It forces you to really think about how all the equations come together, why they work and what they do. Implementing an algorithm is also in a way explaining a model, but in a computer language. You have to understand it or else your computer won’t behave as intended! It also tangentially makes you a better programmer.

One last thing I’d like to mention is that I don’t think feeding your intellectual curiosity should be just about engaging with the literature in your specific field of research. As I’ve mentioned before, good ideas can come from surprising places. Thinking about things outside of your field of research can be fruitful too!

Letting your mind go wild

To me letting the mind go wild is about taking a break, focusing on my hobbies for a little, working out, playing games, socializing, etc. Coming back to a problem with a fresh mind can definitely help you solve it and some research supports this idea. One of the more fruitful ways of letting your mind go wild is probably writing. Putting into writing your thoughts about a specific topic lets your mind loose but in the end you come out with a more structured way of thinking about that topic. It also has the tendency to allow you to identify gaps in your understanding. I think the blog style fits my needs best since it entails sharing my thoughts publicly which makes me more mindful about how clearly I am writing, how structured my thoughts are and so on without being overly formal (and this holds even if you have no actual or substantial audience).

This post is definitely not an exhaustive list of how to improve or acquire skills that will be useful for graduate school and scientific endeavours in general. One of the things I’m really curious about in writing this is how my perspective will shift in the next few years, coming back here to read a young, excited and potentially naive aspiring researcher’s thoughts on science and how to become better at it.

Resources

ML-Brain-Resources

Machine and Deep Learning Compendium

Awesome AGI and CoCoSci

AI Expert Roadmap

Homemade Machine Learning

Machine Learning Experiments