My original plan for the semester was to complete the to-do list given by Dr.Park last semester which included replicating the world models and other model work using Reinforcement Learning. I was trying to go along with my original plan. At the beginning of the semester, I worked on the basic concepts such as Back propagation, Linear Regression, Logistic Regression, clustering etc. Mid-way through the semester Dr.Park advised me that working parallelly on my projects would be more beneficial to me rather than waiting till I felt confident about my basics, as the subject is vast. As suggested, I started working on the to-do list, starting with replicating World Models Carracing. This suggestion was helpful to me and changed my perspective. I found diving into projects and doing relevant reading more beneficial. While I made some progress, I was having problems especially with executing relevant programs with less memory than needed. I met Dr.Park for suggestions and he pointed me to the blogs of students who have already replicated world models and see how it will help me.
In the same meeting, he mentioned that he initially planned to put me in the mixture density network development team but he thinks it would be good for me to work on the environment building team initially. He also advised me to parallelly work on strengthening my concepts. It was good for me to start working on setting up the environment. Initially we started working with retro reflective markers and successfully performed motion tracking with two rigid bodies. We also tried full body skeleton tracking with retro reflective markers. However, we had problems with creating a skeleton and often encountered the error “unable to find skeleton markers”. Initially the problem was due to wrong labelling of markers on our part, marker visibility etc.
Later, we learnt that we will only be using active pucks to do rigid body skeleton tracking. We started experimenting with active pucks. We started facing the same problem with active pucks too- unable to find skeleton markers while trying to create a skeleton. Later a Motive expert came in and addressed a few of our questions. Improved marker visibility, camera angles etc. We were able to create skeletons and refine them properly after a few days. We succeeded in capturing rigid body skeleton mocap and even transferring the mocap onto a Unity rig. Moreover, we also learnt to work with pre-recorded takes. In case of Faceware, we learnt to both implant faceware mocap on Unity rig individually and also to integrate it with motive mocap onto the same rig. However, we are yet to find a way to work with pre-recorded facial mocap takes. The next task is to capture 2 skeleton mocap interacting with rigid body mocap and build a game out of it in Unity.
On a personal level, I feel that I am able to communicate better and that I have become more proactive in my communication. My journal keeping skills have improved over the last semester and I am able to better formulate my tasks. I tend to work in the last minute- a habit that has stayed with me since childhood. This semester, I consciously worked towards prioritizing my work better. In this regard, I feel what Dr.Park has advised me in one of the seminar sessions gave me a solution. He asked me not wait until a project is finished to document the progress. He suggested that it would be more effective and time efficient if I did the documentation parallelly, along with my work. I started implementing what he said and I made good progress in my journaling and documentation work. One more of his suggestions that really helped me was asking me to not keep a list of things to learn later. Though keeping such a list seems like a good idea, in reality, I just procrastinate and end up not reading up on those topics. Instead, he asked me to immediately read up on topics as and when I come across them. I started implementing this by always keeping my laptop open in classes and during seminars. As soon as I come across something that is new to me, I immediately google. This has brought a huge difference in the way and pace that I am learning. Another thing that I started doing has allowed me to retain concepts better; I started to make it a point to always summarize a concept in a one-liner completely written by myself. Now, when I am read those one-line summaries, everything that I read about that topic comes back to me.
As an additional note, though implementing the World Models Carracing no longer constitutes my assigned work, I was determined to complete it and I finally did it towards the end of the semester with a some help from Shadek’ blog and a lot of research on my part.
My experience of working in the HDILab has definitely been great this semester. Being able to make progress in mocap work has definitely boosted my confidence. I also realize that I have become more proactive in my communication especially towards the end of the semester. I hope to continue to be so. I will continue working hard. The lab gives me space to learn and pushes and encourages me to excel. My lab mates are also very helpful and collaborative. The seminars are really informative and helps me become aware of many things. Live coding improve my understanding of code. I hope to be able to proficiently give seminars by myself. Restructuring the seminar topics to establish a common knowledge ground is a good move. I think it will be helpful to me
I am trying to follow the lab culture to the best of my abilities. I try to collaborate with my lab mates whenever possible. I am punctual to the lab and at times when I cannot come on time I make it a point to inform everyone. I am trying to create reproducible output. I am earnest in my work. I have the drive to learn and excel. I think I can improve by producing more blogs.
For the next semester, I plan to complete the 2 skeleton motion capture and build a game in which the model animations (created from motive mocap) can be controlled using user input. I, along with Sanath and Rubel, should start thinking about the logistics and lab setup for when the actual actors come in. I should make a detailed working plan for this after a discussion with Dr.Park., Additionally, I want to be able to publish a paper by the end of my first year. I have to discuss the possibilities with Dr.Park.
Overall, I am happy with how things are progressing right now.