Main Page

Weekly Log: Working at Northwestern University

Main Page Current Project Weekly Log Final Report About My Mentor About Me




  First and Second Week | Third Week | Fourth Week | Fifth Week | Sixth Week | Seventh Week | Eighth Week | Ninth Week | Tenth Week

June 1-12, 2009

Lab Work

         My first assignment as part of joining Dr. Joseph's group was to read up on current literature about the topic at hand. This proved helpful because I gained knowledge into what has already been done in this field. Below is a list of current research on the topic at hand: Later on throughout the week, I was told to start getting a sense of FFMPEG and how to initialize conversions. Since I have had little experience with Ubuntu in the past, I first needed to get a feel on the proper tools and programs I'll need in order to ease myself into using this OS. I needed to know the basic commands through terminal, find how to install packages (which is easy through apt-get), and, when needed, compile, make, and install a non packaged version of a program. Throughout this week I gradually learned this basics of Ubuntu which was a good experience for me because my past internships dealt with Microsoft servers so it's always good to have a little change. I also thank my good friend Chris Hewitt for helping me out whenever I had questions about Ubuntu as he was already experienced on using Linux.

        Now that I knew the basics of the OS and FFMPEG, it was time to start my part of the project, which was to profile FFMPEG and see which MMX procedures were most used when making conversions. This is helpful when trying to manipulate code because if a procedure is only used .01% of the time when encoding or decoding then it would be useless to modify it. At first Dr. Russ Joseph suggested Gprof which FFMPEG can be easily configured with it enabled. When I finally managed to do a test run, it ended up giving pretty good results. Whenever I needed help with configuring any part of these programs or I was stuck on anything, I would either simply Google it or I would go onto to IRC FFMPEG channel where fellow users were able to help me out. Users on this IRC channel as well as comments online suggested I starting using Oprofile to profile FFMPEG instead of gprof. Oprofile is more useful in profiling because it profiles the whole system and does not need programs to be recompiled like Gprof. The only modification needed in the running programs are to enable debug symbols. This is done easily with FFMPEG with just using the --disable-stripping flag when running ./configure in terminal. I then configured Oprofile to filter out all results except debug symbols coming from FFMPEG.

Outside of Work

        The day I got into Chicago, I met up with two new friends I just met a couple weeks before when going to Chicago to scout for apartments. I arrived to Chicago on May 29th to set up my belongings in my apartment early. Luck had it that Northwestern University was actually holding an event that weekend called Dillo day. It is always hosted on the last Saturday of May of every year. Dillo day was a huge event where a large field turned into a stage and various singers performed. It was a great experience and I had lots of fun.

        On Monday, I met Dr. Joseph for the second time and the students that he advised. They had their own lab room and everyone had their own computer. I also met a fellow student, Giang Hoang (G for short), who would be working with me and Dr. Joseph on this project. Both these guys were really nice and got me set up in my own desk. I will be working with them for the next 10 weeks and I'm glad they are really helpful.

        On Friday, I was invited to a lecture by Philip Wadler, a professor of Theoretical physics. The title of his lecture was Well-typed programs can't be blamed. It was a fairly interesting lecture even though some concepts flew over my head at times. Throughout the rest of this week I finalized the approach of what output I need to produce from the profiler. G, one of the students also working under Dr. Joseph, has given me modified code this week so i can compare the profiles of both the unmodified code and the modified error correcting code. Oprofile is very powerful as I can modify the code, recompile it and then run it again and it can compare both profiles and documents its change in running time. I have now started on the bash script to automate these processes and give me the results that I need.





June 15-19, 2009

Lab Work

         This week was concentrated on putting my website together and finishing up my bash script to automate the process of profiling FFMPEG. I put in checks to make sure the user had root access when running the script and also set up a flag, -c, when you just wanted the script to clean up the results it had output. It automatically creates folders and starts profiling when encoding from PPM files to JPEG files. This conversion is done 1000 times that is achieved through a loop in bash script. After this is done, the script shuts down the profiler and outputs the results of the processes and procedures FFMPEG goes through. This process is done using both the unmodified and modified version of FFMPEG on four different .ppm images. Then, the results of the modified and unmodified versions of FFMPEG are compared. Also, a different folder stores the assembly instructions that ran when encoding through FFMPEG. Once I fine tuned everything and everything ran smoothly, I showed and sent the results to G and to Dr. Joseph. As I have mentioned before, I also spent time this week building this website. Now I am going to write bash scripts on decoding from .jpeg to .ppm files and also from .ppm files to different video formats. The video formats include: x264, theora, .3gp, and Xvid.

         Throughout the remainder of the week, I found many test video sequences to use for converting into these tested formats. It was decided that we should put these video formats in the same container format to use throughout all the encoding processes. At first the video test files I found were frames of a video in .ppm files. These weren't ideal since they were in black and white, so I decided to look around for more. I found some good samples from an FTP site by the Video Quality Experts Group(VQEG) and have decided to use those. These are in .YUV formats though and so our methods of testing will have to change. Instead of encoding PPMs to the video formats, we will convert the .yuv files to the different video formats.

Outside of Work

         This week, I got to finally explore the city of Chicago. Since Northwestern University is in Evanston, a suburb of Chicago, making trips by the train took about 45 minutes. This trip was best saved for the weekend. I went to go see Millenium Park, a huge park in the middle of Chicago which holds an outside auditorium and many works of art. I finally got to see the bean and gaze at the beauty of it all. Being here in the middle of Chicago with new friends was an amazing experience. It helps that Northwestern University is on the quarter system and my University is on the semester system. Because of this, I was on break while they were still in school. It made it easier to find new friends that are still in school because no one is gone yet. As it turns out, this week is the beginning of summer vacation for NU students. Most of my friends are going to leave , but, gladly, some live in Chicago. It was really nice meeting new people and one my friends actually graduated.





June 22-26, 2009

Lab Work

         The start to this week was rather different. Instead of coming in at the usual 10:30 AM, I have decided to start my week off with coming in at 8:30 AM to do more in the daylight. Also, there have been problems in encoding the YUV files into other video formats. It seems that the YUV files I have downloaded have no information that FFMPEG can pick up on to convert the files. This includes the framrate, resolution and the format the video was in. It seems these videos were 4:2:2 YUV files with different resolutions and framerate. So I had to test each individual file to make sure I have chosen the right formats in order to convert the video. This was a big hassle and took a good amount of time but I have finished finalizing the parameters I would need when converting these videos into other formats. I have also picked up several other test video sequences from two other sites. This is good because now I have a wide range of different video sequences with different resolutions and framerates. These resolutions range from 176 x 144 to 1920x1080.

         I also dedicated this week to studying the different codecs we'd be using for sound and video encoding. I'm glad I took the time to do this because it further informed me of the codecs we'd be using. There are so many formats out there, it would be hard to choose only a limited set to work with if I did not study some of them. I mainly only focused on the codecs that were popular. FFMPEG is all open source so all the encoding and decoding for this program was made by scratch from developers. It takes time to design and code a codec so any recent codecs that just came out are not likely to be implemented yet. This throws the new blu-ray format (VC-1) out the window. What we have, though, are other many codecs that are wildly popular and others that have been used immensely in the past.

Outside of Work

         I have started asking around Northwestern if there was any way I could get a wildcard, a card students use that gain them access to the bus and gym facilities. As it turns out, since I am in no way affiliated to the school, there is no way to get a wildcard. This would help greatly since I have to either take the train, they call it the el here, or drive my car. It's a 30 minute walk to tech from my apartment but the weather has not been that great. It's beneficial to know what kinds of transportation are at your disposal because it can really help out economically. Also, I have begun to learn to cook for myself because eating out all the time is a huge drain of money. My mother has guided me to begin cooking some homestyle Peruvian cooked meals. It's great learning these meals especially since I'm so far away and can't really eat the meals she makes at home in Florida. Also, since I have a YMCA gym membership in Florida, I could use the gym located in Evanston. Luckily, the gym is only a few blocks away from my apartment. It's always good to start something new when you're in a new city and by yourself.





June 29-July 3, 2009

Lab Work

        The main video codecs that were chosen are H.264, WMV2, Dirac, MPEG1, MPEG2, and MPEG4part2. MPEG1 is one of most popular codecs to date and also a very old one. This codec was meant to compress VHS quality video and a hardware device that can't decode this standard is virtually unheard of. MPEG2 is the successor of MPEG1 is is used everywhere. Its compression technology is used in DVDs and sending video over cable and satellite. WMV2 is an older version of the windows media video format that is popular on the web. WMV3 is the newest format, but it is the same codec as VC-1 so an open source alternative codec has not been developed yet. MPEG4part2 is a newer codec coming from the Moving Picture Experts Group (MPEG). Both DivX and Xvid are implementations of this format and is very popular when ripping video from dvds and other mediums. Ripping is the process of transcoding from a medium like a DVD or Blu-ray disc and storing it onto your hard drive. It is a lossy codec but the file sizes are often much smaller. H.264 is the newest standard that has come from MPEG and includes a more advanced set of video compression techniques. H.264 was made to be highly scalable, encoding videos for phones to encoding blu-ray quality video. Up until now, all these video codecs that were discussed use the discrete cosine transform (DCT) as the main method for compressing video. The only video codec that is different is the Dirac video codec. Dirac actually uses a totally different method than the DCT. Dirac actually uses the discrete wavelet transform(DWT). Dirac is a fairly new codec that just got introduced in 2008 and it would be neat to read on this codec because it is fundamentally different than all the others. As you can see, this week was spent on reading up mostly on these codecs.

Outside of Work

        On Sunday, June 28th, there was a gay pride float parade in the district known as "Boystown." I have never been to a float parade so it was interesting to see and it's always good to experience new things. I went with two friends and saw most of the float parade. It was exciting and often hillarious to see the many different floats. it was a fairly interesting experience and I was glad to go. Also, starting this week, is Taste of Chicago. In this event, restaurants from all around Chicago set up tents in a huge area around Millenium Park and, with tickets, you can have a small taste of what these restaurants have to offer. I went with friends at first and later went with Dr. Joseph and his students. Also, several performers sang in the outside auditorium. Among these artists were most noticeably, Counting Crows and Ne-Yo. It was a great week to be in Chicago.

        Saturday, July 4th, also marked Independence day! Happy Independence Day! Instead of seeing the fireworks in downtown Chicago, a friend and I instead decided to stay in Evanston and look at the fireworks displayed here. Near my apartment, there was a small beach entrance. We went there to see the fireworks and it turned out to be a good time. The town was full of festivities this week.





July 6-10, 2009

Lab Work

        This week, I am spending on making bash shell scripts for video encoding and profiling each instance. I wrote one up beforehand for MPEG1 but now it was time to start work on the other codecs. I figured out that the Synaptic Package Manager on Ubuntu can not be used for installing the codecs or FFMPEG. which is the program I'll be profiling. This is because the version of FFMPEG that comes from package manager has to be freely distributable so all the restricted use codecs were not installed. I instead had to download the source files and compile it myself like before but now I have to add more flags in the configure file. Just in case you were curious, compilation from source are as follows: Once all these flags were set up, I could now encode and decode from the yuv files to the other video formats. I also needed to install the appropiate video codecs. Lucky for me, most of the video codecs are already integrated inside FFMPEG and I just had to install the x264 and Dirac codecs seperately. I also had to configure them seperately and couldn't use the package manager. Again, I needed to enable debugging symbols for both these programs. I used the previous shell script template for encoding and decoding to and from JPEG to PPM to use for my video conversions shell script. Creating this shell script is tougher because there are a lot more conversions happening.

Outside of Work

        Not much went on this week. I learned some more dishes and tried to make it a habit to go work out at least every day. Over the weekend, people play soccer at the fields and I joined in with my co-worker, G. It was great to find other people to play soccer with. Also, the weather has been getting better to play soccer. In the first two weeks in Chicago, the weather was very hot, which I hated because my apartment did not have air conditioning. Never underestimate the power of A/C. I was on the top floor too, so it felt like a sauna without a fan. I expected it to continue being very hot throughout the whole 10 weeks but, to my surprise, it cooled down immensely. The weather would stay aroudn 60 and it would either rain or be very cloudy. This was better than boiling in your own room. I also visited downtown Chicago to see the Sear's Tower. This tower is currently the tallest building in the United States. It cost about 20 dollars to go all the way to the top, known as the sky deck, and it was well worth it. On the top floor, there were compartments that stuck outside the actually building that was made of 3 inch glass. Underneath your feet, you can see the tiny cars and people. This also gave an amazing view of the skyline. On top of that, the sun was setting, which made for an especially beautiful picture of the skyline.





July 13-17, 2009

Lab Work

        I have finally finished making the shell script for the video conversion and tested parts of it. I am having some problems with encoding into some formats and am looking into answers. It seems that each video conversion needs different parameters when encoding and decoding from different codecs. It seems like this part won't be just a simple copy and paste job for the different video codecs. Getting most of these down seemed easy except for x264 and Dirac. The encoding for x264 took a little bit of research on my part to get the proper parameters right but, by the end, the video output finally gave a good quality, non distorted video. For the Dirac video encoding, it seemed that no matter what parameters i put in with the conversion, I still got a greatly distorted video. After some odd hours of scouring through the internet, I finally found a set of decent parameters to give me a decent video output. The only negative aspect that came from these parameters is that timestamps weren't added onto the output video, so you couldn't pause, rewind, or fast foward the video to look for any artifacts.

         On another note, Dr. Joseph asked me to look into the innards of how the video files are actually encoded. I read a few documents and some websites about DCT and DWT but that was just an overview. For each codec itself, these transforms were only a small fraction of what each codec does to encode video files. I looked for documents pertaining to getting an in depth look on these codecs and one of the developers for the x264 codec and ffmpeg program suggested I read a book called H.264 and MPEG-4 Video Compression by Iain E. G. Richardson. I began reading this book and it went fairly in depth on the subject on the H.264 codec without getting into too much technical details. It was fascinating how a encoder actually worked and the neat tricks the encoder implemented in order to reduce space while keeping image quality as well. As it turns out, lossless video compression just reduce redundancies within the data but the size of the file is only modestly reduced. Lossy video compression gains a high amount of compression within videos but some of the actual data within the video will get lost. This means that if you use a lossy compression algorithm, you cannnot decode the encoded video file and expect an exact replica of the video you converted from. This will help us though, because since Dr. Joseph and G are dealing with correcting data calculations with a huge data error to calculations with only a modest amount of error. Using a lossy video compression algorithm can help hide this error even more.

Outside of Work

        It sure is a small world isn't it? I recently met a graduate student from Northwestern University on the train. As it turns out, when I went up to Cornell to visit, he was the one who actually housed me and another student as part of a program. This was amazing that 4 years later, we would see each other again. He also happened to live below me at my apartment. We met and hung out for a few hours one weekend and I got to meet his roommates. I also got to see many new movies with friends this weekend. Some friends and I also drove around Chicago this weeekend to visit communities. We had the chance to visit Humboldt Park, Little Village, and many others. We eventually parked and walked around these communities. Later on, I had the chance to go with friends to see a impov comedy group called Comedy Sportz. In addtion, I got to visit the Art Istitute of Chicago which contained a huge amount of art and a lot of walking. The galleries were huge and after 3 hours of walking around, we have barely seen half the exhibits. Moving closer to the next week, I started missing home a lot since next week will be my birthday and usally I do a family dinner and celebration with them.





July 20-24, 2009

Lab Work

        This week, Dr. Joseph felt the students he advised didn't know each other's projects. He told us by the end of the week to have a short five minute presentation ready to tell each other what we have worked on so far. This task wasn't so bad and I was happy to show what I have learned so far. Reading more on the inner workings of the H.264 codec got me even more interested. As it turns out, for encoding a normal video, the video must first be encoded from the standard RGB color space to the YUV color space. The RGB color space splits the values of the colors within a pixel for red, blue and green. The YUV color space splits the values of the brightness, red, and blue values of each pixel. This color space is more ideal for data compression because humans can perceive brightness (black and white values) more clearly than color values. The pixel values for brightness is kept the same while the color values for each pixel can be downsampled to 1 value per 4 or 2 sets of pixels. The codec then tries to remove as much temporal and spatial redundancy as it can. With temporal redundancy, videos often have the same objects inside many sets of freams because they are shot 30 times a second in a video. This means a lot of the regions inside sequential frames of video have the same object and it would be redundant to save the same objects for each frame. Instead, the encoder saves one frame(I-frame) as a whole in a set of frames and uses pieces of that frame to construct the other frames in that set from it. In case an ojbect in the video moves, only a vector needs to be saved that shows where the object from the I-frame moves instead of saving the object itself. With spatial redundancy, often parts of a frame are copies of other parts of that same frame. Imagine filming a brick wall. For the most part, each brick is the same, and only one brick could be saved in the image. When reconstructing the image, that one brick only needs to be copied over to the rest of the frame. Deleting these redundancies saves a huge amount of space. Just looking for redundancies isn't perfect though, because objects in a video are always moving or turning. The pieces of this frame that are already saved are deleted from the current frame and what you are left with is a residual. This residual then undergoes the DWT or DCT process to further reduce the size of the video. This is just a broad overview of the main processes an encoder does to reduce the size of the file.

        By the end of the week, I finally got the video encoding scripts to fully work. As it turns out, I needed to use GIT and SVN to get the latest codec files and ffmpeg files in the repositories. The repositories for these programs have the most updated versions of these files and contain a huge amount of bug fixes and changes. I installed the latest version of FFMPEG, x264, and Dirac and the problems with them were fixed. X264 added more parameters to help with the encoding process and Dirac can now work without a special workaround to encode a video with timestamps. This week was great. I left the script running over the weekend to profile instances of encoding 1000 times so I can make sure I got all the files needed.

Outside of Work

        This week marks my birthday week. I was born on July 23rd and there were a lot of festivities around. First, my friend surprised me at midnight with a cake and lit candles in hand to wish me a happy birthday. I was working late that day and she surprised me at home. I am happy to have caring friends. Secondly, the Millenium Park outside auditorium hosted a very special group that day. The group's name was NovaLima and they are an Afro-Peruvian band with electronic influences. They were amazing on stage and caused everyone to stand up from their chars and dance in front of the stage. Everyone was happy and joyful and everyone of all ages danced to their music. I guess I lucked out that this coincided with my birthday. Also, later on that night, NovaLima was going to be at an afterparty which me and some friends decided to go to. It was an amazing experience. It was a small bar and it played their music all night long. In the end, I also got their new CD signed by the artists for asnwering a trivia question. It was a good night.





July 27-31, 2009

Lab Work

        This week I was told to start looking into making a list of all the procedures that were called when encoding videos that had any MMX or SSE Intel instructions. These instructions were the main instructions we were looking at since they were SIMD instructions. Our hope was to fix problems that were caused in this section of the CPU and hopefully use this to extend it to fixing hardware problems within graphics processing units (GPU). This was a step into the right direction because all the GPU deals with mostly is SIMD instructions. SIMD instructions work well in multimedia because instead of using one instruction to change a value one at a time, one instruction can be used to change multiple values at a time. For example, if you want to change the brightness of a screen, you could use one instruction to reduce the brightness of six pixels on the screen instead of just one. This greatly reduces the numbers of instructions and also does these changes in parallel. In order to find these instructions, Dr. Joseph suggested I use a python script. I had little knowledge of python but it ultimately proved beneficial. The script takes in the oprofile results file and matches it to a list of procedures that have any MMX or SSE instructions in them. This list of procedures was taken from another file that contains the debugging information from FFMPEG, X264, or Dirac. Once all the procedures that contained these SIMD instructions were found, those procedures were spit out into another file.

        Dr. Joseph also wanted me to look into the C files from FFMPEG and take a procedure that is full of SIMD instructions and describe what is actually happening. He wanted me to create an example inside a well known procedure that codecs use when encoding files. We chose the DCT, IDCT, and DWT transforms. I looked into the file that was run when executing the DCT of a file and it proved to be fairly confusing since most of these procedures are written in inline assembly inside the c file.

Outside of Work

        July 28th marked the Independence day of Peru. On Sunday, July 26th, a huge picnic was hosted by various groups in a large park. This picnic had various Peruvian cuisines that made me miss home even more. I also got to see on stage, the more traditional dances from Peru. People were dancing all around and it felt great to see so many Peruvians in one place. I invited many friends and we had a lot of fun going around the various tents that sold food, music, and clothes. Later this week, my car had an unfortunate problem. It stalled in the middle of the highway and I held up traffic for a good while. It turned out to be an expensive part to replace and the dealer wanted to charge 900 dollars. That was insane, so I found the part myself and installed it myself for only 300 dollars. I'm glad it was easy enough for me to work on. This week is also the second to last week being here and I wanted to stay longer. There were still so many places to see and so many things to do.





August 3-7, 2009

Lab Work

        Finishing up the python and bash scripts this week. Also, finishing up automating the whole process because a lot of files are needed to run FFMPEG along with Oprofile. I added these to the Google group once I was done. I am am trying to organize these files into folders instead of it cluttering the desktop and having a messy organization. I also looked more into the DCT transform. I started commenting each line inside the DCT procedure to get a feel of what the file was doing. The inline assembly was especially difficult since it got down to a more basic language than C. Instead of doing a multiplication of the 8x8 values of the pixel image with the 8x8 DCT matrix, the C file just split the rows and columns of this matrix because it could be solved with less multiplications. Also, the double # symbol inside C was fairly confusing and later on realized it was meant to be replaced by either MMX or SSE2 when executed. C is a very powerful language but also very confusing to comprehend at times. In the end, I gave up what I had. I also spent some time of this week writing my final report.

Outside of Work

         This week marked the last week I'll be here in Chicago. I played soccer with friends one last time and said bye to all my newly acquired friends.My brother also came from Florida to help me pack my belongings and help me drive back down to Virginia. It's a 12 hour ride and I'm glad he was here to help split the driving time. Dr. Joseph invited all his students out to dinner and we had a good time telling each other our goodbyes. Everyone in the lab was very helpful and I couldn't ask for a better environment to work with. This trip definitely opened my eyes to what I want in life, to gain a higher education and experience new places and befriend new people. Sure, you can do that with a job, but as a graduate student, you have much more freedom and most of all, I'm still young and want to live my life to the fullest.