This paper aims to investigate the field of Credit Rating Agencies announcements on stock markets with new ambitions and new methodological tools. Although many event-studies have already been dedicated to this topic, there is a large heterogeneity, first regarding the presence of abnormal returns but also regarding their degree of significance and their reliability. The purpose of this paper is to implement a meta-model to investigate what influences the significance of the market reaction to CRAs’ announcement. Disentangling the investigator’s bias from the “genuine effect” of the CRAs’ announcements requires better identifying the variables that influence the level of significance of the abnormal returns reported in academic literature. We implement a two-stage procedure, relying on logistic regressions based on a sample of 78 articles published in academic journals, covering a publishing period from 1978 to 2021 and a wide range of countries. Firstly, we aim to explain the probability of having significant CAARs by a binary logistic regression. Secondly, through accurate ordered logit models, we aim to know what explains the degree of significance of the CAARs. Indeed, we assume that the reliability of existence of abnormal returns is not identical and depends on the p-value associated with the reported CAARs in literature. We show that the significance of the abnormal returns and their degree of significance is partly explained by the informational content of CRAs announcement and partly by methodological choices made by the authors. Our analysis contributes to disentangle the genuine effect of CRAs’ rating announcements from investigator bias.