Keep my Clang Based tool from analyzing header files in source code?

I'm writing a stand alone tool based on LibTooling with a RecursiveASTVistor, and I don't want to have to go through the contents of header files stored in source code. Do I need to change the compilation database? I've been using this little hack to keep my tool from crashing, but I don't want my tool to analyze any header files at all included in the source code I'm dealing with.

bool VisitDecl(Decl *D) {

    FullSourceLoc FullLocation = Context->getFullLoc(D->getLocStart());
    const auto &SourceManager = Context->getSourceManager();

    if (FullLocation.isValid() && !SourceManager.isInSystemHeader(FullLocation) && Context->getSourceManager().isInMainFile(D->getLocation()))
}