Image overlaps previous view when tapping/swiping back

I have a UIScrollView which covers one of my views entirely. I have added a background image to this same view which scrolls at a slightly different rate to the actual UIScrollView. This works absolutely fine unless I use the back swipe gesture or tap the 'Back' button. What happens is the image covers the view for about 0.5 seconds before disappearing, and it looks pretty bad.

This is what I mean:

As you can see, that is mid way through the gesture, and rather than being able to see the previous view, you just see the part of the image that is off to the left. It doesn't happen on the first page of the UIScrollView so I guess it's because the image is overlapping the previous view.

Here is my code:

override func viewDidLoad() {
    let pagesScrollViewSize = scrollView.frame.size
    scrollView.contentSize = CGSize(width: pagesScrollViewSize.width * CGFloat(images.count), height: pagesScrollViewSize.height)

    backgroundImageView.frame = CGRect(x: 0, y: 0, width: 2484, height: 736)
    backgroundImageView.image = UIImage(named: "SF.png")

    var visualEffectView = UIVisualEffectView(effect: UIBlurEffect(style: .Light)) as UIVisualEffectView
    visualEffectView.frame = backgroundImageView.bounds

    backgroundImageView.addSubview(visualEffectView)

    view.addSubview(backgroundImageView)
    view.sendSubviewToBack(backgroundImageView)

    scrollView.backgroundColor = UIColor.clearColor()
}

func scrollViewDidScroll(scrollView: UIScrollView) {
    loadVisiblePages()

    var factor = scrollView.contentOffset.x / (scrollView.contentSize.width - 414);
    if factor < 0 {
        factor = 0
    }
    if factor > 1 {
        factor = 1
    }

    var frame: CGRect = backgroundImageView.frame
    frame.origin.x = factor * (414 - backgroundImageView.frame.size.width)
    backgroundImageView.frame = frame
}

Anyone have any suggestions?