Following my interests, I made a huge transition in my career a couple of years ago from a marketing career to computer science. There a few people who ardently supported my decision and a lot who questioned it. I was part of both these sets of people. Switching between my fears, doubts and ambition, I continued to work on my skills to pursue this field. Somewhat expectedly, quite surprisingly and to some extent shockingly, I secured a an admission into the Computer Science Doctoral program at UT Arlington.
My interest have loyally stayed with decision making -one of the main reasons to pursue a degree in management.-drawn to fields working on similar stuff After I joined the program at UTA, I met a professor working in area which I can very broadly called decision making but more specifically on reinforcement learning testing the possibility of designing an ‘intelligent agent’ given many rewards including intrinsic social rewards. Apprehensively, I went to meet the professor about a possibility to work with him. In my brief chat with him, I guess I probably made him have some faith in me, for he allowed me to work with him.
Hence began my first semester- Spring 2019
Being my first semester, my plan, broadly, was to find a path towards my PhD, find out what my lab stands for and the final goal it is working for. More specifically, I planned to experiment with the PS4 eye cameras and find the best environment setup for the motion capture to be done at a later stage. Additionally, to support later motion capture work, I decided to also learn working on the ‘Unity’ platform. Besides these activities, I hoped to strengthen my understanding of Machine Learning.
During the semester, I persevered to go with the plan. Hardly into the semester I was assigned to give a seminar on implementing the Character Level RNN designed by Andrej Karpathy. I was excited not only that I would be presenting on a piece of work that I was familiar with but that it’s author is someone that I take inspiration from. The twist, however, was to re-implement the code using TensorFlow, originally written using NumPy. The nightmare, however, was to do a live-coding seminar. Though dauting, given that I haven’t yet become very comfortable with coding, I continued to prepare and in the process, I also became more comfortable with neural net math. Additionally, In order to explain to others in the most simple ways forced me to study and understand at a depth far greater than self-study did. Eventually, my presentation went well.
A month into the semester, as I continued to work on my basics, I started working with the PS4 eye cameras. The equipment setup itself took many days as we were discovering new equipment that we need almost every day. After the initial set up was done, we worked on the calibration, which had to be done several times as the camera calibration was susceptible to even a minor shift in the camera angle. Eventually, we performed motion capture with different combinations such as Multiple Camera Single Person, Multiple camera multiple people etc. We were able to animate an action sequence and converted it into an fbx file. Coming to Unity, I got more comfortable with it as the semester progressed. Along with my other lab mates and Dr.Park, I attended a fortnightly Unity workshop being offered in Dallas . However, after one class , we decided that the workshop was not the right resource for us to be learning Unity. Dr.Park then enrolled us in a few Unity courses from Udemy. These courses were helpful and I found that I learnt more as I replicated what was being presented in the online courses. Once I got a little comfortable with Unity, I was assigned to build a basic game with the fbx files that we generated while experimenting with the cameras. The goal was to design a game in which the animation character performs a variety of actions based on respective user input. After a lot of googling and experimenting, I could design the game as intentioned. Kudos to my lab mate for doing such an interesting array of actions ;) A blog and video for the Unity game are available on my website (which definitely needs a LOT of refinement! ).
Since lab work was mainly focused in the field of Reinforcement Learning, I started to read a lot of Articles related to it. One such article is this explaining the armed bandit problem. While reading this article, I wondered if I could modify and implement the BanditDungeon code in the article to train my Unity game agent to maximize his reward by performing the action with the highest reward. I understand that this is not a complete representation of RL as the rewards will be very sparse. However, I still wanted to try. After a lot of code alteration, I was able to train my agent to learn to perform the most worthwhile action. You can view the trained agent here.
In addition to blog posts and articles, I have also started reading published papers such as ‘The World Models’ and ‘The Gated Architecture’. As an easy way for reference, I have summarized these two papers.
I really enjoyed problem solving. At first I was not confident about my skills. However, as the semester progressed, and I persisted on the problems to be solved, I started seeing results. This gave me more confidence to pursue more difficult tasks.
My biggest challenge this semester has been managing my time and formulating my task list. It felt like there was so much to learn and do. It felt overwhelming and I often floundered to figure out where to start. I am trying to follow the lab culture to the best of my abilities. I would like to work on improving my understanding of AI/ML, producing more output, writing more blogs and reading more Academic papers.
On a personal level, I will need to work on proving my journal keeping skills, time management, task formulation and team collaboration to become a disciplined and serious researcher. Overall, in this semester, my curiosity for learning ML has deepened and the lab definitely helped me find a direction to learn.
I feel like it will be beneficial for me to continue to work in this lab. Humans- the way they think, behave, act- have always been a subject of extreme fascination to me. To be able to work in an environment that is trying to understand human intelligence and replicate it is an honor and is deeply satisfying. Moreover, the students are given a lot of space to learn and experiment. The lab environment is conducive to learning and I really enjoy the collaborative and non-judgmental atmosphere built in the lab. In the same breath, I also feel like be vastly helpful if the student received feedback a little more frequently. Additionally, planning/assigning tasks and seminars much in advance will give us more time to prepare and perform to our satisfaction.
In the coming semester, I plan to work on the to-do list given by Dr.Park