Kogratte,
Sure, $templateCache is a good alternative to test the directive without loading the template. But, at times the template would have certain important elements and you need to make sure that these elements are not modified or, removed from the template so that the directive works as expected. To assert such cases, you need to have tests verifying them against the actual template rather than a mocked response.