Imperial College London
Sample Efficiency, Transfer Learning and Interpretability for Deep Reinforcement Learning
Tell us a bit about yourself – what stage of your PhD are you in and what’s your project about?
I’m a 4th year PhD student, so currently writing up my thesis and working out what to do for the next stage of my career. I work broadly on AI, but within the Department of Bioengineering at my university, so try to incorporate a little inspiration from neuroscience. I’ve collaborated in several different areas of AI, but my main focus is to try and improve the state of intelligent decision making systems (like the control systems for robots). The algorithms that have given the most impressive results, based on a subfield of AI known as deep learning, typically require a large amount of data samples to perform well. To alleviate this, I’ve worked on improving the efficiency of these methods, as well as their ability to transfer from one task to another, which makes them a bit more reusable. Finally, these systems are often considered “black boxes”, so I’ve also looked at how we can better interpret how they form decisions.
What does a typical day or week look like for you?
My typical day starts with responding to emails, catching up with social media, and checking up on issues on GitHub (one of the best parts of my PhD has been contributing to open source research software). Once I finally roll out of bed and make my morning coffee, I have a fairly long commute, so use the opportunity to read a paper. The ratio varies depending on what stage of a research project I’m in, but during the day I’ll be reading papers/books, programming existing/new algorithms, running computational experiments, debugging, analysing results, writing papers, and checking Twitter for what’s the latest in the research community. I also typically have several meetings a week, where I predominantly work with students (from undergraduates to other PhDs) and researchers on various projects. These are mainly in-person, but there’s also the occasional video chat with remote collaborators. On top of that, there are lab meetings, reading groups, research talks and tutorials to attend. At the end of the day, I try to do something active in the evening, go home for dinner, and then relax a bit before going to sleep.
What’s one thing that you’ve enjoyed the most during your PhD?
The best part of doing a PhD is just being able to do what I love. Every day I get to satisfy my intellectual curiosity, learn, and think about really interesting problems. For me, it’s not a 9 to 5 job, it’s what I’m really passionate about. And through it, I’ve made many good friends who share the same passion. Which means I get to hang out and chat with a bunch of really smart and engaging (and nice) people on everything from the latest research to fundamental questions in the field! I promise we also share our lives outside of research, but it’s really nice to have this shared interest.
What’s been the most challenging part of it?
Research is tough – experiments can fail, your work may be scooped by another group, papers get rejected all the time… You may feel pressured to work hard, or even just get absorbed by what you’re doing. And that’s just the PhD – not even considering what you might be going through in your personal life. So one of the major challenges is making sure you look after both your physical and mental health. Everyone has a different “work-life balance” (and for me, “work” is an essential part of my “life”), but it’s important to look after yourself. Maybe certain periods like deadlines will be tough, but it’s good to make sure that most of the time you can eat well, be active, get enough sleep, socialise, and just have the downtime you need.
Where do you see yourself 5 years after completing your PhD?
I’ll definitely be doing research in some capacity. I never planned to do a PhD, but after getting laid off from my first job I decided to return to academia, where I realised that I love being in an environment where I can always be learning more. Whether in 5 years I’ll end up in a big company, a startup, or academia – only time will tell.
What’s one piece of advice that you’d offer students that are thinking of doing a PhD?
I’m going to offer two pieces of advice (sorry)! Firstly, you should be really driven to do a PhD in your chosen area. You will be spending several years, without any particular structure to the day, nor short-term milestones (unless you devise these yourself), working on novel research, so that you can produce a thesis at the end of your PhD. It wouldn’t be research if we knew the outcome, so you need perseverance. Secondly, talk to students in the lab you would like to join about what it’s like there. I chose my lab because I got on very well with my supervisor – he’s very supportive and gives me a lot of freedom. On the other hand, I’ve heard horror stories about other labs, primarily based on the character of the supervisor. Your working environment, and supervisor in particular, will have a huge influence on your PhD.
What makes your university a good place to study?
Imperial College London is one of the top research universities in the world, with some amazing researchers you can work with. Your resources can vary from group to group, but on average they’ll be pretty good. Outside of work, there’s a ridiculous amount of student societies to get involved with, so you can almost certainly find some to fit your current interests, and also try out something new. Finally, London is an exciting and multicultural city, with plenty of places to visit and events to go to.
What do you enjoy doing in your free time?
If I can, I like to be active. I love EDM music and clubbing, and I DJ as a hobby (insert shameless plug here for my DJ duo – Baka Punk – on SoundCloud and Mixcloud). So recently I decided to try something new and actually take dance classes. As a result I now do salsa and bachata – which is good fun – 3 days a week. I also meet up with some old friends once a week to go bouldering. Finally, I’ve been doing martial arts for several years, so do karate (recently got my black belt) and aikijujitsu twice a week, plus kobujutsu and iaijutsu on occasion. I also used to play badminton regularly, but it’s understandably been a bit tricky to fit into my new schedule. For less physical downtime, I usually turn to anime or manga. My current top watch is a sci-fi series called Psycho-Pass – recommended if you’re into the classic sci-fi that is Ghost in the Shell – but for pure relaxation I often go for comedy or slice of life series. I’ve also done plenty of video gaming in the past, having sunk many hours into The Witcher 3 last year, but for now (keeping the music theme) I like to pick up Beat Saber for the occasional quick break.
Want to know more about Kai?
Check out his website and follow him on Twitter using the links below:
Our interviews with PhD students and PhD holders
We’ve interviewed PhD students and PhD holders to ask them what life is like during a PhD, their career development post-PhD and their advice to new students and post-docs
Read more Interviews