If I register the HandleError attribute in global.asax.cs file as below,
public static void RegisterGlobalFilters(GlobalFilterCollection filters)
{
filters.Add(new HandleErrorAttribute());
}
still I need to specify the HandlError attribute in all the controllers?
No, you don't. That's the whole point of global filters.
Global filters run for every action of every controller.