The invariance properties of some well known asymptotic tests are studied. Three types of invariance are considered: invariance to the representation of the null hypothesis, invariance to one-to-one transformations of the parameter space (reparametrizations), and invariance to one-to-one transformations of the model variables such as changes in measurement units. Tests that are not invariant include the Wald test and generalized versions of it, a widely used variant of the Lagrane multiplier test, Neyman's $C(\alpha)$ test, and a generalized version of the latter. For all these tests, we show that simply changing measurement units can lead to vastly different answers even when equivalent null hypotheses are tested. This problem is illustrated by considering regression models with Box-Cox transformations on the variables. We observe, in particular, that various consistent estimators of the information matrix lead to test procedures with different invariance properties. General sufficient conditions are then established, under which the generalized $C(\alpha)$ test becomes invariant to reformulations of the null hypothesis and/or to one-to-one transformations of the parameter space as well as to transformations of the variables. In many practical cases where Wald-type tests lack invariance, we find that special formulations of the generalized $C(\alpha)$ test are invariant and hardly more costly to compute than Wald tests. This computational simplicity stands in contrast with other invariant tests such as the likelihood ratio test. We conclude that noninvariant asymptotic tests should be avoided or used with great care. Further, in many situations, the suggested implementation of the generalized $C(\alpha)$ test often yields an attractive substitute to the Wald test (which is not invariant) and to other invariant tests (which are more costly to perform).