Absolute or relative: the dark side of fund rating systems

Academic literature and market practitioners have always devoted great attention to the analysis of asset management products, with particular regard to fund classification and performance metrics. Less attention has been paid to rating methodologies and to the risk of attributing positive ratings to underperforming asset managers. The most widespread rating criterion is the ordinal one, which is based on the assumption that the best asset managers are those who have performed better than their competitors regardless of their ability to achieve a given threshold (i.e. a positive overperformance against the benchmark). Our study, after a description of the most common risk-adjusted performance measures, introduces the idea of attributing the rating on a cardinal basis, setting in advance a given threshold that should be achieved to receive a positive evaluation (i.e. a rating equal to or higher than 3 on a scale of 1–5). The empirical test conducted on a sample of funds (belonging to the main equity and bond asset classes) made it possible to quantify the e®ects of the cardinal approach on the attribution of the rating and on the probability of assigning a good rating to underperforming funds. Empirical analysis also highlighted how the cardinal method allows, on average, better performance than the ordinal one even in an out-of-sample framework. The differences between the two methodologies are particularly remarkable in efficient markets such as the North American equity market. The two rating assignment systems were also analyzed using contingency tables to test the ability to anticipate the default event (underperformance relative to the benchmark). The policy suggestion emerging from our study concerns the significant impact of the rating criterion in reducing the risk of recommending funds that, despite a good rating, have failed to perform satisfactorily and are unlikely to do so in the future either.