Facial Expression Recognition - Data Preprocessing

Kaggle - Challenges in Representation Learning: Facial Expression Recognition Challenge

https://www.kaggle.com/c/challenges-in-representation-learning-facial-expression-recognition-challenge/data

The data consists of 48x48 pixel grayscale images of faces. The faces have been automatically registered so that the face is more or less centered and occupies about the same amount of space in each image. The task is to categorize each face based on the emotion shown in the facial expression in to one of seven categories (0=Angry, 1=Disgust, 2=Fear, 3=Happy, 4=Sad, 5=Surprise, 6=Neutral).

train.csv contains two columns, "emotion" and "pixels". The "emotion" column contains a numeric code ranging from 0 to 6, inclusive, for the emotion that is present in the image. The "pixels" column contains a string surrounded in quotes for each image. The contents of this string a space-separated pixel values in row major order. test.csv contains only the "pixels" column and your task is to predict the emotion column.

The training set consists of 28,709 examples. The public test set used for the leaderboard consists of 3,589 examples. The final test set, which was used to determine the winner of the competition, consists of another 3,589 examples.

This dataset was prepared by Pierre-Luc Carrier and Aaron Courville, as part of an ongoing research project. They have graciously provided the workshop organizers with a preliminary version of their dataset to use for this contest.

In [1]:
%matplotlib inline
In [2]:
import sys
sys.path.append('/Users/rahulsridhar/anaconda2/lib/python2.7/site-packages') # To import cv2, which datagenerator uses
In [3]:
import numpy as np
import matplotlib.pyplot as plt
import matplotlib.gridspec as gridspec
from os import listdir
from os.path import isfile, join
import datagenerator
import cv2
import pickle
import os
In [4]:
np.random.seed(15)

Create datasets

In [4]:
# Create training data
with open('train.txt','w') as f:
    for i in range(7):
        path = 'Training/'+str(i)
        onlyfiles = [fl for fl in listdir(path) if isfile(join(path, fl))]
        print len(onlyfiles), onlyfiles[0]
        for curr_file in onlyfiles:
            if curr_file.find(".DS_Store") != -1:
                continue
            fname = path + '/' + curr_file + ' ' + str(i) + '\n'
            f.write(fname)
3995 Training_10002922.jpg
436 Training_10191968.jpg
4097 Training_10069035.jpg
7214 Training_10001243.jpg
4830 Training_10023337.jpg
3171 Training_10081105.jpg
4964 Training_10002382.jpg
In [5]:
# Create public test data
with open('test.txt','w') as f:
    for i in range(7):
        path = 'PublicTest/'+str(i)
        onlyfiles = [fl for fl in listdir(path) if isfile(join(path, fl))]
        print len(onlyfiles), onlyfiles[0]
        for curr_file in onlyfiles:
            if curr_file.find(".DS_Store") != -1:
                continue
            fname = path + '/' + curr_file + ' ' + str(i) + '\n'
            f.write(fname)
467 PublicTest_10113400.jpg
56 PublicTest_1123625.jpg
496 PublicTest_1023506.jpg
895 PublicTest_100405.jpg
653 PublicTest_10308692.jpg
415 PublicTest_100155.jpg
607 PublicTest_10110281.jpg
In [6]:
# Create private test data
with open('pri_test.txt','w') as f:
    for i in range(7):
        path = 'PrivateTest/'+str(i)
        onlyfiles = [fl for fl in listdir(path) if isfile(join(path, fl))]
        print len(onlyfiles), onlyfiles[0]
        for curr_file in onlyfiles:
            if curr_file.find(".DS_Store") != -1:
                continue
            fname = path + '/' + curr_file + ' ' + str(i) + '\n'
            f.write(fname)
491 PrivateTest_10097204.jpg
55 PrivateTest_10754785.jpg
528 PrivateTest_10022244.jpg
879 PrivateTest_10055093.jpg
594 PrivateTest_10147544.jpg
416 PrivateTest_10106550.jpg
626 PrivateTest_10297218.jpg
In [9]:
# Verify that all files have been written to the text files
with open("train.txt", 'r') as f1, open("test.txt", 'r') as f2, open("pri_test.txt", 'r') as f3:
    train_lines = f1.readlines()
    test_lines = f2.readlines()
    pri_test_lines = f3.readlines()
    print len(train_lines), len(test_lines), len(pri_test_lines)
28707 3589 3589
In [10]:
# Delete objects that are not required
del train_lines, test_lines, pri_test_lines, onlyfiles, f, f1, f2, fl, i, path

Compute image mean

In [ ]:
# Read all filenames
with open('train.txt','r') as f:  
    allfiles = f.readlines()

Takes a lot of time

In [73]:
# Initialization
im_sum = np.zeros((48, 48)) 
In [ ]:
# Compute cumulative sum of normalized images
im_sum = np.zeros((48, 48))
for i in range(len(allfiles)):
    #print i, 
    filename = allfiles[i].split()[0]
    currFile = plt.imread(filename)
    currCopy = currFile
    
    # Normalize the image
    norm_image = cv2.normalize(src=currFile, dst=currCopy, alpha=0, beta=1, \
                               norm_type=cv2.NORM_MINMAX, dtype=cv2.CV_32F)
    im_sum += norm_image
os.system('say "Your program has finished"')    
In [78]:
# Ensure that the loop has run till the end and verify object shapes
print i 
print im_sum.shape
print im_sum
print "Min max", np.min(im_sum), np.max(im_sum) # Ensure that there are no NaNs
28706
(48, 48)
[[ 13967.36620638  13586.33595832  13215.78351716 ...,  13099.92431601
   13460.37629166  13816.13901912]
 [ 13820.33572791  13425.8313489   13075.99294613 ...,  12970.62055342
   13337.559397    13694.18310627]
 [ 13644.40881324  13271.9466602   12932.99909543 ...,  12840.80047006
   13202.16191077  13561.30518893]
 ..., 
 [ 13655.90575565  13533.51676058  13417.79948726 ...,  13078.58731053
   13184.66140734  13293.28373564]
 [ 13662.52940704  13538.10444834  13412.52978735 ...,  13086.22241009
   13185.66268183  13293.34675919]
 [ 13663.11242045  13529.50784723  13384.21543764 ...,  13082.61723414
   13176.75306863  13273.76755988]]
Min max 12039.7259336 19901.991298
In [79]:
# Compute image mean
im_mean = im_sum/len(allfiles) 
In [82]:
print im_mean
print "Mean min max", np.min(im_mean), np.max(im_mean)
[[ 0.48654914  0.47327606  0.46036798 ...,  0.45633206  0.4688883
   0.48128119]
 [ 0.48142738  0.46768493  0.45549841 ...,  0.4518278   0.46461     0.47703289]
 [ 0.47529901  0.4623244   0.45051726 ...,  0.44730555  0.45989347
   0.47240412]
 ..., 
 [ 0.47569951  0.47143612  0.46740514 ...,  0.45558879  0.45928385
   0.46306767]
 [ 0.47593024  0.47159593  0.46722158 ...,  0.45585475  0.45931873
   0.46306987]
 [ 0.47595055  0.47129647  0.46623525 ...,  0.45572917  0.45900836
   0.46238783]]
Mean min max 0.419400353 0.693280081443
In [85]:
np.save("Training_Img_Mean", im_mean) # Store object

Check - normalization doesn't modify the images

In [84]:
# Read a sample image
temp = plt.imread(allfiles[0].split()[0])
temp2 = temp
In [65]:
# Normalize the image
norm_image = cv2.normalize(src=temp, dst=temp2, alpha=0, beta=1, norm_type=cv2.NORM_MINMAX, dtype=cv2.CV_32F)
In [49]:
plt.imshow(temp, cmap='gray') # Original image
Out[49]:
<matplotlib.image.AxesImage at 0x10912c2d0>
In [67]:
plt.imshow(norm_image, cmap='gray') # Normalized image
Out[67]:
<matplotlib.image.AxesImage at 0x1093659d0>
In [ ]: