Measuring economic journals’ citation efficiency: a data envelopment analysis approach

Abstract

This paper by using data envelopment analysis (DEA) and statistical inference evaluates the citation performance of 229 economic journals. The paper categorizes the journals into four main categories (A–D) based on their efficiency levels. The results are then compared to the 27 “core economic journals” as introduced by Diamond (Curr Contents 21(1):4–11, 1989). The results reveal that after more than 20 years Diamonds’ list of “core economic journals” is still valid. Finally, for the first time the paper uses data from four well-known databases (SSCI, Scopus, RePEc, Econlit) and two quality ranking reports (Kiel Institute internals ranking and ABS quality ranking report) in a DEA setting and in order to derive the ranking of 229 economic journals. The ten economic journals with the highest citation performance are Journal of Political Economy, Econometrica, Quarterly Journal of Economics, Journal of Financial Economics, Journal of Economic Literature, American Economic Review, Review of Economic Studies, Journal of Econometrics, Journal of Finance, Brookings Papers on Economic Activity.

Appendix: methodology and statistical techniques applied

Based on the work by Koopmans (1951) and Debreu (1951) the production set Ψ constraints the production process and is the set of physically attainable points (x, y):

$$ \Uppsi = \left\{ {\left( {x,y} \right) \in \left. {\Re_{ + }^{N + M} } \right|x\;{\text{can}}\;{\text{produce}}\;y} \right\} $$

(4)

where $x \in \Re_{ + }^{N}$ is the input vector and $y \in \Re_{ + }^{M}$ is the output vector. For the input oriented efficiency score a country operating at the level (x, y) is defined as:

$$ \theta \left( {x,y} \right) = { \inf }\left\{ {\left. \theta \right|\left( {\theta x,y} \right) \in \Uppsi } \right\} $$

(5)

Furthermore, DEA became more popular when introduced by Charnes et al. (1978) in order to estimate Ψ allowing for constant returns to scale (CRS model). Later, Banker et al. (1984) introduced a DEA estimator allowing for variable returns to scale (VRS model). In our case, when evaluating journals’ citation performance input orientation of DEA models have been applied due to the fact that input quantities appear to be the primary decision variables (Coelli and Perelman 1999; Coelli et al. 2005; Halkos and Tzeremes 2010). The quality of the papers appeared in a journal but also the number of the papers to be published (i.e. the number of issues and volumes) is subject to the editors’ decision. Therefore the decision makers have most control over the input compared to the outputs used. Furthermore, the CRS model developed by Charnes et al. (1978) can be calculated as:

$$ \hat{\Uppsi }_{\text{CRS}} = \left\{ {\begin{array}{*{20}c} {\left( {x,y} \right) \in \left. {\Re^{N + M} } \right|y \le \sum\limits_{i = 1}^{n} {\gamma_{i} y_{i} ;x} \ge \sum\limits_{i = 1}^{n} {\gamma_{i} x_{i} } \;{\text{for}}\;\left( {\gamma_{1} , \ldots \gamma_{n} } \right)} \hfill \\ {{\text{such}}\;{\text{that}}\;\gamma_{i} \ge 0,\;i = 1, \ldots n} \hfill \\ \end{array} } \right\} $$

(6)

The VRS model developed by Banker et al. (1984) allowing for variable returns to scale can then be calculated as:

$$ \hat{\Psi }_{{{\text{VRS}}}} = \left\{ {\begin{array}{*{20}c} {\left( {x,y} \right) \in \Re ^{{N + M}} \left| {y \le \sum\limits_{{i = 1}}^{n} {\gamma _{i} y_{i} ;x} \ge \sum\limits_{{i = 1}}^{n} {\gamma _{i} x_{i} } } \right.\;{\text{for}}\;\left( {\gamma _{1} , \ldots \gamma _{n} } \right)} \hfill \\ {{\text{such}}\;{\text{that}}\;\sum\limits_{{i = 1}}^{n} {\gamma _{i} = 1} ,\;\gamma _{i} \ge 0,\;i = 1, \ldots n} \hfill \\ \end{array} } \right\} $$

(7)

Then in order to obtain the corresponding input oriented DEA estimators of efficiency scores we need to plug in $\hat{\Uppsi }_{\text{CRS}}$ and $\hat{\Uppsi }_{\text{VRS}}$ respectively in Eq. 5 presented previously.

Simar and Wilson (1998, 2000, 2008) suggest that DEA estimators were shown to be biased by construction. They introduced an approach based on bootstrap techniques (Efron 1979) to correct and estimate the bias of the DEA efficiency indicators. The bootstrap bias estimate for the original DEA estimator $\hat{\theta }_{\text{DEA}} (x,y)$can be calculated as:

$$ \widehat{\text{BIAS}}_{B} \left( {\hat{\theta }_{\text{DEA}} (x,y)} \right) = B^{ - 1} \sum\limits_{b = 1}^{B} {\hat{\theta }_{{{\text{DEA}},b}}^{*} (x,y) - \hat{\theta }_{\text{DEA}} (x,y)} $$

(8)

Furthermore, $\hat{\theta }_{{{\text{DEA}},b}}^{*} (x,y)$ are the bootstrap values and B is the number of bootstrap replications. Then a biased corrected estimator of θ(x, y) can be calculated as:

$$ \begin{aligned} \hat{\hat{\theta }}_{\text{DEA}} (x,y) = & \hat{\theta }_{\text{DEA}} (x,y) - \widehat{\text{BIAS}}_{B} \left( {\hat{\theta }_{\text{DEA}} (x,y)} \right) \\ = & 2\hat{\theta }_{\text{DEA}} (x,y) - B^{ - 1} \sum\limits_{b = 1}^{B} {\hat{\theta }_{{{\text{DEA}},b}}^{*} (x,y)} \\ \end{aligned} $$

(9)

However, according to Simar and Wilson (2008) this bias correction can create an additional noise and the sample variance of the bootstrap values $\hat{\theta }_{{{\text{DEA}},b}}^{*} (x,y)$ need to be calculated. The calculation of the variance of the bootstrap values is illustrated below:

$$ \hat{\sigma }^{2} = B^{ - 1} \sum\limits_{b = 1}^{B} {\left[ {\hat{\theta }_{{{\text{DEA}},b}}^{*} (x,y) - B^{ - 1} \sum\limits_{b = 1}^{B} {\hat{\theta }_{{{\text{DEA}},b}}^{*} (x,y)} } \right]^{2} } $$

(10)

We need to avoid the bias correction illustrated in (9) unless:

$$ \frac{{\left| {\widehat{\text{BIAS}}_{B} (\hat{\theta }_{\text{DEA}} (x,y))} \right|}}{{\hat{\sigma }}} > \frac{1}{\sqrt 3 } $$

(11)

Following Shephard (1970) the input distance function can be expressed as $\hat{\delta }_{\text{DEA}} \left( {x,y} \right) \equiv \frac{1}{{\hat{\theta }_{\text{DEA}} \left( {x,y} \right)}}$ then we can construct bootstrap confidence intervals for $\hat{\delta }_{\text{DEA}} \left( {x,y} \right)$ as:

$$ \left[ {\hat{\delta }_{\text{DEA}} \left( {x,y} \right) - \hat{\alpha }_{1 - \alpha /2} ,\hat{\delta }_{\text{DEA}} \left( {x,y} \right) - \hat{\alpha }_{\alpha /2} } \right] $$

(12)

In order to choose between the adoption of the results obtained by the CCR (Charnes et al. 1978) and BCC (Banker et al. 1984) models in terms of the consistency of our results obtained we adopt the method introduced by Simar and Wilson (2002). Therefore, we compute the DEA efficiency scores under the CRS and VRS assumption and by using the bootstrap algorithm we test for the CRS results against the VRS results obtained such as:

$$ H_{ 0} :\;\Uppsi^{\vartheta } \;{\text{is}}\;{\text{CRS}}\;{\text{against}}\;H_{1} :\;\Uppsi^{\vartheta } \;{\text{is}}\;{\text{VRS}} $$

(13)

The test statistic is given as:

$$ T\left( {X_{n} } \right) = \frac{1}{n}\sum\limits_{i = 1}^{n} {\frac{{\hat{\theta }_{{{\text{CRS}},n}} \left( {X_{i} ,Y_{i} } \right)}}{{\hat{\theta }_{{{\text{VRS}},n}} \left( {X_{i} ,Y_{i} } \right)}}} $$

(14)

Then the p-value of the null hypotheses can be approximated by the proportion of bootstrap samples as:

$$ p{\text{-value}} = \sum\limits_{b = 1}^{B} {\frac{{I\left( {T^{ * ,b} \le T_{\text{obs}} } \right)}}{B}} $$

(15)

where B is 2000 bootstrap replications, I is the indicator function and $T^{*,b}$ is the bootstrap samples and original observed values are denoted by T _obs.