Skip to main content
Log in

Not all areas are equal: analysis of citations in information security research

  • Published:
Scientometrics Aims and scope Submit manuscript

Abstract

The understanding of the inner workings of a research community is essential to evaluate the impact of an author as well as to decide where and how to publish results. One of the key metrics is the number of citations that a publication receives. In parallel, information security is now a key and strategic area, partially fueled by the advent of the Internet of Things (IoT) and the need of pursuing cybercriminals by using digital forensics techniques. Therefore, this paper analyzes several factors influencing the number of citations in the domain of information security, such as differences between journal and conference publications, or the impact of the number of pages and the length of the abstract. To obtain quantitative results, we investigated papers of six sub-disciplines, i.e., anonymity and privacy, cryptography, information hiding, IoT and Cyber-Physical System security, digital forensics and incident response, and network security. For each sub-domain, we used metadata of 5000 publications collected from IEEE-Xplore. Results indicate some clear behaviors, for instance, papers tend to receive more citations when their abstract is longer and the number of references positively influences the performance of the work.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8

Similar content being viewed by others

Notes

  1. http://ieee-xplore.ieee.org/Xplore/home.jsp

  2. Fernandes and Monteiro reported a continuous per-decade increase in the number of authors per paper in information security research since the mid-1970s (Fernandes and Monteiro 2017). As a side-effect of our analysis, were able to confirm their observation.

  3. https://dl.acm.org/

  4. http://citeseerx.ist.psu.edu/index

  5. https://www.researchgate.net/

References

  • Anderson, T. (2009). Conference reviewing considered harmful. ACM SIGOPS Operating Systems Review, 43(2), 108–116. https://doi.org/10.1145/1531793.1531815.

    Article  Google Scholar 

  • Baldwin, J., Alhawi, O. M., Shaughnessy, S., Akinbi, A., & Dehghantanha, A. (2018). Emerging from the cloud: A bibliometric analysis of cloud forensics studies. In A. Dehghantanha, M. Conti, & T. Dargahi (Eds.), Cyber threat intelligence (pp. 311–331). Cham: Springer.

    Chapter  Google Scholar 

  • Bornmann, L., & Leydesdorff, L. (2018). Count highly-cited papers instead of papers with h citations: Use normalized citation counts and compare “like with like”!. Scientometrics, 115(2), 1119–1123. https://doi.org/10.1007/s11192-018-2682-1.

    Article  Google Scholar 

  • Burcham, M., Al-Zyoud, M., Carver, J. C., Alsaleh, M., Du, H., Gilani, F., Jiang, J., Rahman, A., Kafalı, O., Al-Shaer, E., & Williams, L. (2017). Characterizing scientific reporting in security literature: An analysis of ACM CCS and IEEE S&P papers. In Proceedings of the hot topics in science of security: Symposium and bootcamp, HoTSoS (pp. 13–23). ACM, New York. https://doi.org/10.1145/3055305.3055307

  • Cabaj, K., Caviglione, L., Mazurczyk, W., Wendzel, S., Woodward, A., & Zander, S. (2018). The new threats of information hiding: the road ahead. IT Professional, 20(3), 31–39.

    Article  Google Scholar 

  • Carver, J. C., Burcham, M., Kocak, S. A., Bener, A., Felderer, M., Gander, M., King, J., Markkula, J., Oivo, M., Sauerwein, C., & Williams, L. (2016). Establishing a baseline for measuring advancement in the science of security: An analysis of the 2015 IEEE security and privacy proceedings. In Proceedings of the symposium and bootcamp on the science of security, HotSos’16 (pp. 38–51). ACM, New York. https://doi.org/10.1145/2898375.2898380

  • Cavelty, M. D. (2018). Cybersecurity research meets science and technology studies. Politics and Governance, 6(2), 22–30.

    Article  Google Scholar 

  • Chakraborty, J., Pradhan, D., Dutta, H. S., Nandi, S., & Chakraborty, T. (2018). On good and bad intentions behind anomalous citation patterns among journals in computer sciences. CoRR arXiv:1807.10804

  • Chakraborty, T. (2018). Role of interdisciplinarity in computer sciences: Quantification, impact and life trajectory. Scientometrics, 114(3), 1011–1029. https://doi.org/10.1007/s11192-017-2628-z.

    Article  Google Scholar 

  • Cobo, M., Jürgens, B., Herrero-Solana, V., Martínez, M., & Herrera-Viedma, E. (2018). Industry 4.0: A perspective based on bibliometric analysis. Procedia Computer Science, 139, 364–371. https://doi.org/10.1016/j.procs.2018.10.278. (6th International Conference on Information Technology and Quantitative Management).

    Article  Google Scholar 

  • Collberg, C., & Proebsting, T. A. (2016). Repeatability in computer systems research. Communications of the ACM, 59(3), 62–69. https://doi.org/10.1145/2812803.

    Article  Google Scholar 

  • Corporation, M. (2010). The science of cyber security. Tech. Rep. JSR-10-102, MITRE. http://www.fas.org/irp/agency/dod/jason/cyber.pdf

  • Fernandes, J. M., & Monteiro, M. P. (2017). Evolution in the number of authors of computer science publications. Scientometrics, 110, 529–539.

    Article  Google Scholar 

  • Fiala, D., & Tutoky, G. (2017). Computer science papers in web of science: A bibliometric analysis. Publications, 5(4), 23. https://doi.org/10.3390/publications5040023.

    Article  Google Scholar 

  • Goodrum, A. A., McCain, K. W., Lawrence, S., & Giles, C. L. (2001). Scholarly publishing in the internet age: A citation analysis of computer science literature. Information Processing & Management, 37(5), 661–675.

    Article  Google Scholar 

  • Herley, C., & van Oorschot, P. C. (2017). Sok: Science, security and the elusive goal of security as a scientific pursuit. In 2017 IEEE symposium on security and privacy (SP) (pp. 99–120). IEEE. https://doi.org/10.1109/SP.2017.38

  • Humayed, A., Lin, J., Li, F., & Luo, B. (2017). Cyber-physical systems security-a survey. IEEE Internet of Things Journal, 4(6), 1802–1831.

    Article  Google Scholar 

  • Iqbal, W., Qadir, J., Tyson, G., Mian, A. N., Hassan, Su, & Crowcroft, J. (2019). A bibliometric analysis of publications in computer networking research. Scientometrics, 119(2), 1121–1155. https://doi.org/10.1007/s11192-019-03086-z.

    Article  Google Scholar 

  • Isenberg, P., Isenberg, T., Sedlmair, M., Chen, J., & Möller, T. (2017). Visualization as seen through its research paper keywords. IEEE Transactions on Visualization and Computer Graphics, 23(1), 771–780. https://doi.org/10.1109/TVCG.2016.2598827.

    Article  Google Scholar 

  • Jia, H., & Saule, E. (2017). An analysis of citation recommender systems: Beyond the obvious. In Proceedings of the 2017 IEEE/ACM international conference on advances in social networks analysis and mining 2017, ASONAM ’17 (pp. 216–223). ACM, New York, NY. https://doi.org/10.1145/3110025.3110150

  • Kott, A. (2014). Towards fundamental science of cyber security. In R. E. Pino (Ed.), Network science and cybersecurity (pp. 1–13). New York: Springer.

    Google Scholar 

  • Kousha, K., & Thelwall, M. (2007). Google scholar citations and google web/url citations: A multi-discipline exploratory analysis. Journal of the Association for Information Science and Technology, 58(7), 1055–1065.

    Google Scholar 

  • Li, M. (2018). Classifying and ranking topic terms based on a novel approach: Role differentiation of author keywords. Scientometrics, 116(1), 77–100.

    Article  Google Scholar 

  • MacDonald, K. I., & Dressler, V. (2018). Using citation analysis to identify research fronts: A case study with the internet of things. Science & Technology Libraries, 37(2), 171–186. https://doi.org/10.1080/0194262X.2017.1415183.

    Article  Google Scholar 

  • Makawana, P. R., & Jhaveri, R. H. (2018). A bibliometric analysis of recent research on machine learning for cyber security. In Y. C. Hu, S. Tiwari, K. K. Mishra, & M. C. Trivedi (Eds.), Intelligent Communication and Computational Technologies (pp. 213–226). Singapore: Springer Singapore.

    Chapter  Google Scholar 

  • Mazurczyk, W., & Caviglione, L. (2015). Information hiding as a challenge for malware detection. IEEE Security Privacy, 13(2), 89–93.

    Article  Google Scholar 

  • Meyer, B. (2011). The nastiness problem in computer science. Posting in the blog of Comm. ACM. http://cacm.acm.org/blogs/blog-cacm/123611-the-nastiness-problem-in-computer-science/fulltext

  • Mogul, J. C. (2013). Towards more constructive reviewing of CS papers. ACM SIGCOMM Computer Communication Review, 43(3), 90–94. https://doi.org/10.1145/2500098.2500112.

    Article  Google Scholar 

  • Muhamedyev, R. I., Aliguliyev, R. M., Shokishalov, Z. M., & Mustakayev, R. R. (2018). New bibliometric indicators for prospectivity estimation of research fields. Annals of Library and Information Studies, 65, 62–69.

    Google Scholar 

  • Shen, S., Rousseau, R., & Wang, D. (2018). Do papers with an institutional e-mail address receive more citations than those with a non-institutional one? Scientometrics, 115, 1039–1050.

    Article  Google Scholar 

  • Stanton, J., Mastrangelo, P., Stam, K., & Jolton, J. (2004). Behavioral information security: Two end user survey studies of motivation and security practices. In AMCIS 2004 proceedings (pp. 175).

  • Sun, X., Ding, K., & Lin, Y. (2016). Mapping the evolution of scientific fields based on cross-field authors. Journal of Informetrics, 10(3), 750–761.

    Article  Google Scholar 

  • Vieira, E. S., & Gomes, J. A. (2009). Citations to scientific articles: Its distribution and dependence on the article features. Journal of Informetrics, 4, 1–13.

    Article  Google Scholar 

  • Wendzel, S. (2018). Get me cited, Scotty! Analysis of academic publications in covert channel/steganography research. In Proceedings of the 13th international conference on availability, reliability and security (ARES’18) (pp. 13:1–13:8). ACM, New York, NY. https://doi.org/10.1145/3230833.3233265

  • Yan, B. N., Lee, T. S., & Lee, T. P. (2015). Mapping the intellectual structure of the internet of things (IoT) field (2000–2014): a co-word analysis. Scientometrics, 105(2), 1285–1300.

    Article  Google Scholar 

  • Yang, K., & Meho, L. I. (2006). Citation analysis: A comparison of google scholar, scopus, and web of science. Proceedings of the Association for Information Science and Technology, 43(1), 1–15.

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Steffen Wendzel.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Wendzel, S., Lévy-Bencheton, C. & Caviglione, L. Not all areas are equal: analysis of citations in information security research. Scientometrics 122, 267–286 (2020). https://doi.org/10.1007/s11192-019-03279-6

Download citation

  • Received:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11192-019-03279-6

Keywords

Navigation