Training SVM in OpenCV

Currently I am working on a program which should detect people on a livestream in real-time. Therefore I am using C++ with OpenCV 3.1. My approach is to train SVM and use HOG Detector.

I think something is going wrong when i train the SVM, especially by building the trainingmatrix. My Detector loads the created SVM and uses it with hog.detectMultiscale(). Using the default-svm from opencv 'getDefaulPeopleDetector' I got some good results on easy pictures.

I am in the need to apply my self-trained SVM because our testing setup seems to be pretty difficult and the default one doesn't cover it. The results I got by executing my detector are really bad, f.e. detects 'people' on a covered webcam/black screen or random detections throughout the entire screen.

Here is my code:

getImages loads every picture in a directory and checks if its size is coorect (64x128)

    //Reads and checks the images from the given directory
static void getImages(vector<string>& fileNames, const string dirName) {
    DIR *dir;
    dir = opendir(dirName.c_str());
    struct dirent *ent;
    string imgName;
    string imgPath;
    Mat img;


    if (dir != NULL) {
        while ((ent = readdir (dir)) != NULL) {
        imgName = ent->d_name;
            imgPath = dirName + "/" +imgName;
        img = imread(imgPath);
          if(!img.data) { 
        cout << imgPath << " has no data" << endl;    
          } else {
          if(img.rows != IMG_ROW_SIZE && img.cols != IMG_COL_SIZE) {
            cout << imgName << " size not correct: " << img.cols << "x" << img.rows <<endl;
          } else {
            cout << imgPath << " loaded" << endl;
            fileNames.push_back(imgPath);
          }
          }
        }
        closedir (dir);
    } else {
        cout << dirName << " not present" << endl;
    } 
}

In this method I create the training matrix which is used by svm.train(). For every image I compute the hog Features. One feature vector is inserted in one row of the training matrix. So every row contains the features of one image. I am not sure if the computation of the hogfeatures is correctly implemented or if the matrix isn't created in the way shown in this post (SVM with images)

static void initTrainingMats(Mat& trainingMat, vector<string>& positiveImages, vector<string>& negativeImages, Mat& labelMat) {
HOGDescriptor hog;
vector<float>  hogFeature;
Mat img;

cout << "computing hog features.. ";
for(int i = 0; i<trainingMat.rows; i ++) { // for-loop iterating through all images
img = (i < positiveImages.size() ? imread(positiveImages.at(i)) : imread(negativeImages.at(i-positiveImages.size())));
labelMat.at<float>(i,1) = (i < positiveImages.size() ? 1 : -1);
hog.compute(img, hogFeature, Size(8,8), Size(8,8));
for(int j = 0; j<trainingMat.cols; j++) {
  trainingMat.at<float>(i,j) = hogFeature.at(j);      
}
}

This is my main method. Initializing the SVM and saving it.

int main(int, char**)
{
    //vectors to store file path to each correct image
    vector<string> positiveImages;
    vector<string> negativeImages;
    getImages(positiveImages, posDirName);
    getImages(negativeImages, negDirName);
    int allImages = positiveImages.size() + negativeImages.size();

    //initialize training matrix and label matrix
    Mat trainingMat(allImages, 3780 ,CV_32FC1); //matrix with column size same as size of hog_sample vector
    Mat labelMat(allImages, 1, CV_32SC1);
    initTrainingMats(trainingMat, positiveImages, negativeImages, labelMat);


    Ptr<SVM> svm = SVM::create(); 
    svm->setType(SVM::C_SVC);
    svm->setKernel(SVM::LINEAR);
    svm->setTermCriteria(TermCriteria(TermCriteria::MAX_ITER, 100, 1e-10));
    svm->train(trainingMat, ROW_SAMPLE, labelMat);
    svm->save("../../hog_video/build/svm_v2");

    cout<<"SVM stored successfully" << endl;
    }

I am kinda stuck here, I am glad for every help/suggestions you guys got for me!