Torumoy Ghoshal
ABSTRACT
Recently convolutional neural networks (CNN) have shown exceptional performance with data where a feature structure is explicitly defined, for example image data. Real world data is often represented as d dimensional vectors and they lack such feature structure. If features could be embedded into a low dimensional space to introduce feature locality, CNNs could take advantage of the newly introduced feature structure and show better performance. In this paper, we present a technique of feature embedding to introduce feature locality so that non-image data exhibit image like feature structure. We achieve this by embedding features into a 1d or 2d space using t-SNE. We show that CNN performs better under the proposed approach.
- M. M. Bronstein, J. Bruna, Y. LeCun, A. Szlam, and P. Vandergheynst. 2017. Geometric Deep Learning: Going beyond Euclidean Data. IEEE Signal Processing Magazine 34, 4 (July 2017), pp. 18--42.Google Scholar
- D. C. Ciresan, U. Meier, and J. Schmidhuber. 2012. Multi-column Deep Neural Networks for Image Classification. 2012 IEEE Conference on Computer Vision and Pattern Recognition (2012), pp. 3642--3649. Google ScholarDigital Library
- D. Dua and C. Graff. 2017. UCI Machine Learning Repository. http://archive.ics.uci.edu/mlGoogle Scholar
- Y. Kim. 2014. Convolutional Neural Networks for Sentence Classification. In Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing, EMNLP 2014, October 25-29, 2014, A meeting of SIGDAT, a Special Interest Group of the ACL. Doha, Qatar, pp. 1746--1751. http://aclweb.org/anthology/D/D14/D14-1181.pdfGoogle ScholarCross Ref
- A. Krizhevsky, I. Sutskever, and G. Hinton. 2012. ImageNet Classification with Deep Convolutional Neural Networks. In Proceedings of the 25th International Conference on Neural Information Processing Systems - Volume 1 (NIPS'12). Curran Associates Inc., Lake Tahoe, Nevada, USA, pp. 1097--1105. http://dl.acm.org/citation.cfm?id=2999134.2999257 Google ScholarDigital Library
- Y. LeCun, Y. Bengio, and G. Hinton. 2015. Deep learning. Nature 521 (27 May 2015), pp. 436--444.Google Scholar
- Y. LeCun, B.Boser, J. S. Denker, D. Henderson, R. E. Howard, W. Hubbard, and L. D. Jackel. 1989. Backpropagation Applied to Handwritten Zip Code Recognition. Neural Computation 1, 4 (Winter 1989), pp. 541--551. Google ScholarDigital Library
- Y. LeCun and C. Cortes. 2010. MNIST handwritten digit database. http://yann.lecun.com/exdb/mnist/. (2010). http://yann.lecun.com/exdb/mnist/Google Scholar
- T. Mikolov, I. Sutskever, K. Chen, G. S. Corrado, and J. Dean. 2013. Distributed Representations of Words and Phrases and their Compositionality. In Advances in Neural Information Processing Systems 26, C.J. C. Burges, L. Bottou, M. Welling, Z. Ghahramani, and K. Q. Weinberger (Eds.). Curran Associates, Inc., pp. 3111--3119. Google ScholarDigital Library
- J. Pennington, R. Socher, and C. D. Manning. 2014. GloVe: Global Vectors forWord Representation. In Empirical Methods in Natural Language Processing (EMNLP). Doha, Qatar, pp. 1532--1543. http://www.aclweb.org/anthology/D14-1162Google Scholar
- L.J.P. van der Maaten and G. Hinton. 2008. Visualizing High-Dimensional Data Using t-SNE. Journal of Machine Learning Research 9 (2008), pp. 2579--2605.Google Scholar
Index Terms
- Improving Performance of Convolutional Neural Networks via Feature Embedding
Recommendations
Semi-supervised local multi-manifold Isomap by linear embedding for feature extraction
We explore the discriminative feature extraction problem.A SemiSupervised local multimanifold Isomap by linear embedding is proposed.Our model can use labeled and unlabeled data to deliver manifold features.Our model aims to minimize pairwise intraclass ...
Feature learning for steganalysis using convolutional neural networks
Traditional steganalysis methods usually rely on handcrafted features. However, with the rapid development of advanced steganography, manual design of complex features has become increasingly difficult. In this paper, we propose a new paradigm for ...
Deep Convolutional Neural Networks for Feature Extraction of Images Generated from Complex Networks Topologies
To identify topology features of different complex network topology is essential in network science researches. Apart from traditional tools in doing such jobs such as power-law, a proved method of convolutional neural network (CNN) is introduced into ...
Comments