DD4T Default route not working

I am using the DD4T framework with SDL Tridion 2011 and ASP.NET MVC4

I have some non-DD4T Controllers and Views which I want to use, however I am getting a 404 error when I try to go to these urls.

This is my route table

    public static void RegisterRoutes(RouteCollection routes)
    {
        routes.IgnoreRoute("{resource}.axd/{*pathInfo}");

        // Route for SDL Tridion UI 2011 (Aka !SiteEdit)
        routes.MapRoute(
            "SiteEditBlankPage",
            "se_blank.html",
            new { controller = "Empty", action = "Index" });

        routes.MapRoute(
            null, // Route name
            "xml/{*PageId}", // URL with parameters
            new { controller = "Page", action = "Xml" }, // Parameter defaults
           new { pageId = @"^(.*)?$" } // Parameter constraints
        );

        // Tridion page route
        routes.MapRoute(
           "TridionPage",
           "{*PageId}",
           new { controller = "Page", action = "Page" }, // Parameter defaults
           new { pageId = @"^(.*)?$" } // Parameter constraints
        );

        routes.MapRoute(
            null, // Route name
            "{controller}/{action}/{id}", // URL with parameters
            new { controller = "Home", action = "Index", id = UrlParameter.Optional } // Parameter defaults
        );
    }

I have tried moving the default route above the Tridion one but then I get 404s for the Tridion pages.

The only way to get it working it seems is to have a specific route to my controller e.g:

        routes.MapRoute(
           null, // Route name
           "MyController/{action}/{id}", // URL with parameters
           new { controller = "MyController", action = "Index", id = UrlParameter.Optional } // Parameter defaults
       );

Anyone have any ideas? As the above solution is not ideal.

(I do not have any UI 2012 config in my web.config)

There are several options AFAICS:

1. If you can recognize a Tridion url from a 'normal' url (extension, startpath, etc) you can solve it by adjusting the TridionPage route. But this is no option for you I think, because if you can recognize the Tridion url's from the normal one's you already would have come up with this solution ;)

2. Implement a route-constraint on the TridionPage route. Check in this routeconstraint if the requested url is in the Broker. If true: return true, otherwise false. If it returns false, the next route (that matches) will handle the request. Not sure about performance though.

3. Your own option: a specific route for your normal pages.

I am sure I missed some options. Hopefully someone else shares them here.