Accelerate - Lower the threshold for contrast stretching

I am using Swift and Accelerate and am trying to color-correct an image using the vImageContrastStretch method available in Accelerate's vImage module.

When I try to stretch the histograms I get a result which does do exactly what I want, but it is a bit too relaxed. The resulting image histograms still have unwanted space on the sides. As you can see in the image, the result of the vImageContrastStretch (middle) operation still has some room for snipping on both sides of the histogram (marked red). The result I am aiming for is on the right - notice the histogram differences.

How can I snip the red parts of the histogram as well? Do I have to write my own algorithm or is there a simpler solution using Accelerate?

Code sample that I am using to generate the result:

import UIKit
import PlaygroundSupport
import CoreImage
import Accelerate

let bitmapInfo:CGBitmapInfo = CGBitmapInfo(
    rawValue: CGImageAlphaInfo.last.rawValue)

var format = vImage_CGImageFormat(
    bitsPerComponent: 8,
    bitsPerPixel: 32,
    colorSpace: nil,
    bitmapInfo: bitmapInfo,
    version: 0,
    decode: nil,
    renderingIntent: CGColorRenderingIntent.defaultIntent)

extension CIImage
{
    convenience init?(fromvImageBuffer: vImage_Buffer)
    {
        var mutableBuffer = fromvImageBuffer
        var error = vImage_Error()

        let cgImage = vImageCreateCGImageFromBuffer(
            &mutableBuffer,
            &format,
            nil,
            nil,
            UInt32(kvImageNoFlags),
            &error)

        self.init(cgImage: cgImage!.takeRetainedValue())
    }
}

let inputImage: UIImage = UIImage(named: "IMG_2225.JPG")!

let ciImage = CIImage(cgImage: inputImage.cgImage!)

let imageRef = CIContext().createCGImage(
    ciImage,
    from: ciImage.extent)

var imageBuffer = vImage_Buffer()

vImageBuffer_InitWithCGImage(
    &imageBuffer,
    &format,
    nil,
    imageRef!,
    UInt32(kvImageNoFlags))

let pixelBuffer = malloc(imageRef!.bytesPerRow * imageRef!.height)

var outBuffer = vImage_Buffer(
    data: pixelBuffer,
    height: UInt(imageRef!.height),
    width: UInt(imageRef!.width),
    rowBytes: imageRef!.bytesPerRow)

vImageContrastStretch_ARGB8888(
    &imageBuffer,
    &outBuffer,
    UInt32(kvImageNoFlags))

let outImage = CIImage(fromvImageBuffer: outBuffer)
free(imageBuffer.data)
free(pixelBuffer)

outImage!

It seems, from the description, that vImageEndsInContrastStretch_ARGB8888() offers the capability you're looking for. From the Histogram overview in the docs:

Ends-in contrast stretch is a more complex version of the contrast stretch operation. These types of functions are best used on images that have some pixels at or near the lowest and highest values of the intensity spectrum, but whose histogram is still mainly concentrated in one area. The ends-in contrast stretch functions map all intensities less than or equal to a certain level to 0; all intensities greater than or equal to a certain level to 255; and perform a contrast stretch on all the values in between. The low and high levels are not defined directly by two given intensity values, but by percentages: the ends-in contrast stretch operation must find intensity levels such that a certain percent of pixels are below one of the intensity values, and a certain percent are above the other intensity value.