Looking at this the examples show refspecs can filter directories/namespaces as in:

+refs/heads/qa/*:refs/remotes/origin/qa/*

However I'm trying to optimise a fetch and the repo I'm working with has branches named like this:

release-6.0.0

When trying to filter these I get:

fatal: Invalid refspec '+refs/heads/release*:refs/remotes/upstream/release*'

Is there any way to filter these in the fetch or do I need to fetch all the remote heads?

Git 2.6+ (Q3 2015) will allow this kind of refspec!

See commit cd377f4, commit 53a8555 (22 Jul 2015) by Jacob Keller (jacob-keller).
^{(Merged by Junio C Hamano -- gitster -- in commit 8d3981c, 03 Aug 2015)}

refs: loosen restriction on wildcard "*" refspecs

Loosen restrictions on refspecs by allowing patterns that have a "*" within a component instead of only as the whole component.

Remove the logic to accept a single "*" as a whole component from check_refname_format(), and implement an extended form of that logic in check_refname_component().
Pass the pointer to the flags argument to the latter, as it has to clear REFNAME_REFSPEC_PATTERN bit when it sees "*".

Teach check_refname_component() function to allow an asterisk "*" only when REFNAME_REFSPEC_PATTERN is set in the flags, and drop the bit after seeing a "*", to ensure that one side of a refspec contains at most one asterisk.

This will allow us to accept refspecs such as for/bar*:foo/baz*.
Any refspec which functioned before shall continue functioning with the new logic.

---

Original answer (2013)

It doesn't seem possible, because that limitation facilitates refs matching done by remote.c.

That goes back to Git 1.5.6.5 (August 2008) and commit b2a5627 (Daniel Barkalow) which enforces the rule of "wildcard refspec must end with slash and star":

A wildcard refspec is internally parsed into a refspec structure with src and dst strings.
Many parts of the code assumed that these do not include the trailing "/*" when matching the wildcard pattern with an actual ref we see at the remote.
What this meant was that we needed to make sure not just that the prefix matched, and also that a slash followed the part that matched.

But a codepath that scans the result from ls-remote and finds matching refs forgot to check the "matching part must be followed by a slash" rule.
This resulted in "refs/heads/b1" from the remote side to mistakenly match the source side of "refs/heads/b/*:refs/remotes/b/*" refspec.

Worse, the refspec crafted internally by "git-clone", and a hardcoded preparsed refspec that is used to implement "git-fetch --tags", violated this "parsed widcard refspec does not end with slash" rule; simply adding the "matching part must be followed by a slash" rule then would have broken codepaths that use these refspecs.

This commit changes the rule to require a trailing slash to parsed wildcard refspecs.
IOW, "refs/heads/b/*:refs/remotes/b/*" is parsed as src = "refs/heads/b/" and dst = "refs/remotes/b/".
This allows us to simplify the matching logic because we only need to do a prefixcmp() to notice "refs/heads/b/one" matches and "refs/heads/b1" does not.

---

The OP Monte Goulding points out to commit 46220ca (diff) as being the origin of this rule (Git 1.5.5, April 2007)

We tightened the refspec validation code in an earlier commit ef00d15 (Tighten refspec processing, 2008-03-17) per my suggestion, but the suggestion was misguided to begin with and it broke this usage:

$ git push origin HEAD~12:master

The syntax of push refspecs and fetch refspecs are similar in that they are both colon separated LHS and RHS (possibly prefixed with a + to force), but the similarity ends there.
For example, LHS in a push refspec can be anything that evaluates to a valid object name at runtime (except when colon and RHS is missing, or it is a glob), while it must be a valid-looking refname in a fetch refspec.
To validate them correctly, the caller needs to be able to say which kind of refspecs they are.
It is unreasonable to keep a single interface that cannot tell which kind it is dealing with, and ask it to behave sensibly.

This commit separates the parsing of the two into different functions, and clarifies the code to implement the parsing proper (i.e. splitting into two parts, making sure both sides are wildcard or neither side is).