How do I pass a class as an argument to a function in UnrealScript?

I want to do

result = TraceActors(class'QuadForcePawn', HitEnemy, hitLocation, HitNorm, weaponStart, m_oldWeaponStartLocation[iHand], vExtent);

but I want to substitute a variable for the class'QuadForcePawn' part. I haven't been able to figure out how to do this.

This is so I can have a function that calls TraceActors, and that function takes an argument telling it which kinds of Actors to look for.

Technically, TraceActors is actually an iterator function and is meant to be used with the foreach keyword, so you wouldn't actually assign the result to a variable.

To answer your question, what you want to use is a class reference variable. An example function might look like this (TraceActors actually requires many more parameters, but I've left them out for brevity.):

function TraceFor (class<Actor> traceClass)
{
    local Actor A;

    foreach TraceActors(traceClass, A)
    {
        // do work here
    }
}

Class reference variables are declared with the class keyword, and optionally you can use the <> syntax to limit the classes which can be assigned to it. class<Actor> declares a class reference variable which can only have Actor or subclasses of Actor assigned to it. In the example function above, a call to TraceFor(class'Pawn') would work fine, but a call to TraceFor(class'Object') would fail to compile.

If you want to return the result of the TraceActors call, one way to do it might be to build an array of the results and return that:

// Returns an array of Actors of the passed in type.
function TraceFor (class<Actor> traceClass, out array<Actor> results)
{
    local Actor A;

    foreach TraceActors(traceClass, A)
    {
        results.AddItem(A);
    }
}