PLDANet: Reasonable Combination of PCA and LDA Convolutional Networks
DOI:
https://doi.org/10.15837/ijccc.2022.2.4541Keywords:
deep learning, principal component analysis, fisher principal component analysis, convolutional coverageAbstract
Integrating deep learning with traditional machine learning methods is an intriguing research direction. For example, PCANet and LDANet adopts Principal Component Analysis (PCA) and Fisher Linear Discriminant Analysis (LDA) to learn convolutional kernels separately. It is not reasonable to adopt LDA to learn filter kernels in each convolutional layer, local features of images from different classes may be similar, such as background areas. Therefore, it is meaningful to adopt LDA to learn filter kernels only when all the patches carry information from the whole image. However, to our knowledge, there are no existing works that study how to combine PCA and LDA to learn convolutional kernels to achieve the best performance. In this paper, we propose the convolutional coverage theory. Furthermore, we propose the PLDANet model which adopts PCA and LDA reasonably in different convolutional layers based on the coverage theory. The experimental study has shown the effectiveness of the proposed PLDANet model.
References
[1] Ahmed, S.B.; Ali, S.F.; Ahmad, J.; Adnan, M.; and Fraz, M.M. (2020). On the frontiers of pose invariant face recognition: a review. Artificial Intelligence Review, 53(4), 2571-2634, 2020. https://doi.org/10.1007/s10462-019-09742-3
[2] Alzyoud, F.Y.; Maqableh, W.; and Faiz Al, S. (2021). A semi smart adaptive approach for trash classification. INTERNATIONAL JOURNAL OF COMPUTERS COMMUNICATIONS & CONTROL, 16(4172), 1-13, 2021. https://doi.org/10.15837/ijccc.2021.4.4172
[3] Arun, K.S.; Govindan, V.K.; and Kumar, S.D.M. (2020). Enhanced bag of visual words representations for content based image retrieval: a comparative study. Artificial Intelligence Review, 53(3), 1615-1653, 2020. https://doi.org/10.1007/s10462-019-09715-6
[4] Cao, Z.; Duan, L.; Yang, G.; Yue, T.; and Chen, Q. (2019). An experimental study on breast lesion detection and classification from ultrasound images using deep learning architectures. BMC Medical Imaging, 19(1), 511-519, 2019. https://doi.org/10.1186/s12880-019-0349-x
[5] Carrara, F.; Falchi, F.; Caldelli, R.; Amato, G.; Fumarola, R.; and Becarelli, R. (2017). Detecting adversarial example attacks to deep neural networks. 1-7, 2017. https://doi.org/10.1145/3095713.3095753
[6] Carreira-Perpián, M.. and Raziperchikolaei, R. (2015). Hashing with binary autoencoders. In 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2015. https://doi.org/10.1109/CVPR.2015.7298654
[7] Chan, T.H.; Jia, K.; and Gao, S. (2015). Pcanet: A simple deep learning baseline for image classification? IEEE TRANSACTIONS ON IMAGE PROCESSING, 24, 5017-5032, 2015. https://doi.org/10.1109/TIP.2015.2475625
[8] Dlv, S. and Sor, R. (2007). An empirical evaluation of deep architectures on problems with many factors of variation. In 24th International Conference on Machine Learning. 473-480, 2007.
[9] Groth, D.; Hartmann, S.; Klie, S.; and Selbig, J. (2013). Principal components analysis. Methods Mol Biol, 930, 527-547, 2013. https://doi.org/10.1007/978-1-62703-059-5_22
[10] Huang, G.; Liu, Z.; Pleiss, G.; Maaten, L.V.D.; and Weinberger, K. (2019). Convolutional networks with dense connectivity. IEEE transactions on pattern analysis and machine intelligence, 1(4), 1-12, 2019. https://doi.org/10.1109/TPAMI.2019.2918284
[11] Jing, L.; Tao, Q.; Chang, W.; Kai, X.; and Fang-Qing, W. (2018). Robust face recognition using the deep c2d-cnn model based on decision-level fusion. Sensors, 18(7), 2080, 2018. https://doi.org/10.3390/s18072080
[12] Krizhevsky, A.; Sutskever, I.; and Hinton, G.E. (2017). Imagenet classification with deep convolutional neural networks. Communications of the ACM, 60(6), 84-90, 2017. https://doi.org/10.1145/3065386
[13] Lecun, Y. and Bottou, L. (1998). Gradient-based learning applied to document recognition. Proceedings of the IEEE, 86(11), 2278-2324, 1998. https://doi.org/10.1109/5.726791
[14] Lee, D.T. and Lin, A.K. (1986). Generalized delaunay triangulation for planar graphs. Discrete & Computational Geometry, 1, 201-217, 1986. https://doi.org/10.1007/BF02187695
[15] Lintang, R.A.; Purnawarman, M.; and Wibowo, E.P. (2017). Human face recognition application using pca and eigenface approach. In 2017 Second International Conference on Informatics and Computing (ICIC), 2017.
[16] Liong, V.E.; Lu, J.; and Wang, G. (2013). Face recognition using deep pca. In 2013 9th International Conference on Information, Communications & Signal Processing (ICICS). 1-5, 2013. https://doi.org/10.1109/ICICS.2013.6782777
[17] Liu, C.; Zhang, T.; Ding, D.; and Lv, C. (2016). Design and application of compound kernel-pca algorithm in face recognition. In 2016 35th Chinese Control Conference (CCC). 4122-4126, 2016. https://doi.org/10.1109/ChiCC.2016.7553997
[18] Lu, J.; Liong, V.E.; Wang, G.; and Moulin, P. (2017). Joint feature learning for face recognition. IEEE Transactions on Information Forensics and Security, 10(7), 1371-1383, 2017. https://doi.org/10.1109/TIFS.2015.2408431
[19] Machidon, A.L.; Machidon, O.M.; and Ogrutan, P.L. (2019). Face recognition using eigenfaces, geometrical pca approximation and neural networks. In 2019 42nd International Conference on Telecommunications and Signal Processing (TSP), 2019. https://doi.org/10.1109/TSP.2019.8768864
[20] Ooyen, A.V. and Nienhuis, B. (1992). Improving the convergence of the back-propagation algorithm. Neural Networks, 5, 465-471, 1992. https://doi.org/10.1016/0893-6080(92)90008-7
[21] Oziuddeen, M.A.K.; Poruran, S.; and Caffiyar, M.Y. (2020). A novel deep convolutional neural network architecture based on transfer learning for handwritten urdu character recognition. Tehnicki vjesnik, 27(4), 1160-1165, 2020. https://doi.org/10.17559/TV-20190319095323
[22] Pereira, J.F.; Barreto, R.M.; Cavalcanti, G.D.C.; and Tsang, I.R. (2011). A robust feature extraction algorithm based on class-modular image principal component analysis for face verification. In 2011 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). 1469-1472, 2011. https://doi.org/10.1109/ICASSP.2011.5946770
[23] Riffenburgh, R. and Clunies-Ross, C. (1960). Linear discriminant analysis. Pacific Science, 14, 251-256, 1960.
[24] Riffenburgh, R.H. and Clunies-Ross, C.W. (2013). Linear discriminant analysis. Pacific Science, 3(6), 27-33, 2013.
[25] Shin, H.C.; Roth, H.R.; Gao, M.; Lu, L.; Xu, Z.; Nogues, I.; Yao, J.; Mollura, D.; and Summers, R.M. (2016). Deep convolutional neural networks for computer-aided detection: Cnn architectures, dataset characteristics and transfer learning. IEEE Transactions on Medical Imaging, 35(5), 1285-1298, 2016. https://doi.org/10.1109/TMI.2016.2528162
[26] Stuhlsatz, A.; Lippel, J.; and Zielke, T. (2014). Feature extraction with deep neural networks by a generalized discriminant analysis. IEEE Transactions on Neural Networks and Learning Systems, 23(4), 596-608, 2014. https://doi.org/10.1109/TNNLS.2012.2183645
[27] Sun, K.; Zhang, J.; Yong, H.; and Liu, J. (2018). Fpcanet: Fisher discrimination for principal component analysis network. Knowledge-Based Systems, 166, 108-117, 2018. https://doi.org/10.1016/j.knosys.2018.12.015
[28] Swati, Z.N.K.; Zhao, Q.; Kabir, M.; Ali, F.; and Lu, J. (2019). Brain tumor classification for mr images using transfer learning and fine-tuning. Computerized Medical Imaging and Graphics, 75(7), 34-46, 2019. https://doi.org/10.1016/j.compmedimag.2019.05.001
[29] Unar, S.; Wang, X.; Wang, C.; and Wang, Y. (2019). A decisive content based image retrieval approach for feature fusion in visual and textual images. Knowledge-Based Systems, 179, 8-20, 2019. https://doi.org/10.1016/j.knosys.2019.05.001
[30] Xiaofeng; Qi; Lei; Zhang; Yao; Chen; Yong; Pi; Yi; and and, Q. (2018). Automated diagnosis of breast ultrasonography images using deep neural networks. Medical Image Analysis, 52, 185-198, 2018. https://doi.org/10.1016/j.media.2018.12.006
[31] Z., L.; M., P.; and S.Z., L. (2014). Learning discriminant face descriptor for face recognition. IEEE Transactions on Pattern Analysis and Machine Intelligence, 36(2), 289-302, 2014. https://doi.org/10.1109/TPAMI.2013.112
[32] Zhang, C.; Mei, Z.; Wu, B.; Yu, J.; andWang, Q. (2020). Query with assumptions for probabilistic relational databases. Tehnicki vjesnik, 27(3), 923-932, 2020. https://doi.org/10.17559/TV-20191123110408
[33] Zhang, S.; Yang, L.T.; Feng, J.; Wei, W.; Cui, Z.; Xie, X.; and Yan, P. (2021). A tensor-networkbased big data fusion framework for cyber-physical-social systems (cpss). Information Fusion, (76), 337-354, 2021. https://doi.org/10.1016/j.inffus.2021.05.014
[34] Zhou, Y.; Wang, Y.; and Wang, X.H. (2018). Face recognition algorithm based on wavelet transform and local linear embedding. Cluster Computing, 22, 1529-1540, 2018. https://doi.org/10.1007/s10586-018-2157-4
[2] Alzyoud, F.Y.; Maqableh, W.; and Faiz Al, S. (2021). A semi smart adaptive approach for trash classification. INTERNATIONAL JOURNAL OF COMPUTERS COMMUNICATIONS & CONTROL, 16(4172), 1-13, 2021. https://doi.org/10.15837/ijccc.2021.4.4172
[3] Arun, K.S.; Govindan, V.K.; and Kumar, S.D.M. (2020). Enhanced bag of visual words representations for content based image retrieval: a comparative study. Artificial Intelligence Review, 53(3), 1615-1653, 2020. https://doi.org/10.1007/s10462-019-09715-6
[4] Cao, Z.; Duan, L.; Yang, G.; Yue, T.; and Chen, Q. (2019). An experimental study on breast lesion detection and classification from ultrasound images using deep learning architectures. BMC Medical Imaging, 19(1), 511-519, 2019. https://doi.org/10.1186/s12880-019-0349-x
[5] Carrara, F.; Falchi, F.; Caldelli, R.; Amato, G.; Fumarola, R.; and Becarelli, R. (2017). Detecting adversarial example attacks to deep neural networks. 1-7, 2017. https://doi.org/10.1145/3095713.3095753
[6] Carreira-Perpián, M.. and Raziperchikolaei, R. (2015). Hashing with binary autoencoders. In 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2015. https://doi.org/10.1109/CVPR.2015.7298654
[7] Chan, T.H.; Jia, K.; and Gao, S. (2015). Pcanet: A simple deep learning baseline for image classification? IEEE TRANSACTIONS ON IMAGE PROCESSING, 24, 5017-5032, 2015. https://doi.org/10.1109/TIP.2015.2475625
[8] Dlv, S. and Sor, R. (2007). An empirical evaluation of deep architectures on problems with many factors of variation. In 24th International Conference on Machine Learning. 473-480, 2007.
[9] Groth, D.; Hartmann, S.; Klie, S.; and Selbig, J. (2013). Principal components analysis. Methods Mol Biol, 930, 527-547, 2013. https://doi.org/10.1007/978-1-62703-059-5_22
[10] Huang, G.; Liu, Z.; Pleiss, G.; Maaten, L.V.D.; and Weinberger, K. (2019). Convolutional networks with dense connectivity. IEEE transactions on pattern analysis and machine intelligence, 1(4), 1-12, 2019. https://doi.org/10.1109/TPAMI.2019.2918284
[11] Jing, L.; Tao, Q.; Chang, W.; Kai, X.; and Fang-Qing, W. (2018). Robust face recognition using the deep c2d-cnn model based on decision-level fusion. Sensors, 18(7), 2080, 2018. https://doi.org/10.3390/s18072080
[12] Krizhevsky, A.; Sutskever, I.; and Hinton, G.E. (2017). Imagenet classification with deep convolutional neural networks. Communications of the ACM, 60(6), 84-90, 2017. https://doi.org/10.1145/3065386
[13] Lecun, Y. and Bottou, L. (1998). Gradient-based learning applied to document recognition. Proceedings of the IEEE, 86(11), 2278-2324, 1998. https://doi.org/10.1109/5.726791
[14] Lee, D.T. and Lin, A.K. (1986). Generalized delaunay triangulation for planar graphs. Discrete & Computational Geometry, 1, 201-217, 1986. https://doi.org/10.1007/BF02187695
[15] Lintang, R.A.; Purnawarman, M.; and Wibowo, E.P. (2017). Human face recognition application using pca and eigenface approach. In 2017 Second International Conference on Informatics and Computing (ICIC), 2017.
[16] Liong, V.E.; Lu, J.; and Wang, G. (2013). Face recognition using deep pca. In 2013 9th International Conference on Information, Communications & Signal Processing (ICICS). 1-5, 2013. https://doi.org/10.1109/ICICS.2013.6782777
[17] Liu, C.; Zhang, T.; Ding, D.; and Lv, C. (2016). Design and application of compound kernel-pca algorithm in face recognition. In 2016 35th Chinese Control Conference (CCC). 4122-4126, 2016. https://doi.org/10.1109/ChiCC.2016.7553997
[18] Lu, J.; Liong, V.E.; Wang, G.; and Moulin, P. (2017). Joint feature learning for face recognition. IEEE Transactions on Information Forensics and Security, 10(7), 1371-1383, 2017. https://doi.org/10.1109/TIFS.2015.2408431
[19] Machidon, A.L.; Machidon, O.M.; and Ogrutan, P.L. (2019). Face recognition using eigenfaces, geometrical pca approximation and neural networks. In 2019 42nd International Conference on Telecommunications and Signal Processing (TSP), 2019. https://doi.org/10.1109/TSP.2019.8768864
[20] Ooyen, A.V. and Nienhuis, B. (1992). Improving the convergence of the back-propagation algorithm. Neural Networks, 5, 465-471, 1992. https://doi.org/10.1016/0893-6080(92)90008-7
[21] Oziuddeen, M.A.K.; Poruran, S.; and Caffiyar, M.Y. (2020). A novel deep convolutional neural network architecture based on transfer learning for handwritten urdu character recognition. Tehnicki vjesnik, 27(4), 1160-1165, 2020. https://doi.org/10.17559/TV-20190319095323
[22] Pereira, J.F.; Barreto, R.M.; Cavalcanti, G.D.C.; and Tsang, I.R. (2011). A robust feature extraction algorithm based on class-modular image principal component analysis for face verification. In 2011 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). 1469-1472, 2011. https://doi.org/10.1109/ICASSP.2011.5946770
[23] Riffenburgh, R. and Clunies-Ross, C. (1960). Linear discriminant analysis. Pacific Science, 14, 251-256, 1960.
[24] Riffenburgh, R.H. and Clunies-Ross, C.W. (2013). Linear discriminant analysis. Pacific Science, 3(6), 27-33, 2013.
[25] Shin, H.C.; Roth, H.R.; Gao, M.; Lu, L.; Xu, Z.; Nogues, I.; Yao, J.; Mollura, D.; and Summers, R.M. (2016). Deep convolutional neural networks for computer-aided detection: Cnn architectures, dataset characteristics and transfer learning. IEEE Transactions on Medical Imaging, 35(5), 1285-1298, 2016. https://doi.org/10.1109/TMI.2016.2528162
[26] Stuhlsatz, A.; Lippel, J.; and Zielke, T. (2014). Feature extraction with deep neural networks by a generalized discriminant analysis. IEEE Transactions on Neural Networks and Learning Systems, 23(4), 596-608, 2014. https://doi.org/10.1109/TNNLS.2012.2183645
[27] Sun, K.; Zhang, J.; Yong, H.; and Liu, J. (2018). Fpcanet: Fisher discrimination for principal component analysis network. Knowledge-Based Systems, 166, 108-117, 2018. https://doi.org/10.1016/j.knosys.2018.12.015
[28] Swati, Z.N.K.; Zhao, Q.; Kabir, M.; Ali, F.; and Lu, J. (2019). Brain tumor classification for mr images using transfer learning and fine-tuning. Computerized Medical Imaging and Graphics, 75(7), 34-46, 2019. https://doi.org/10.1016/j.compmedimag.2019.05.001
[29] Unar, S.; Wang, X.; Wang, C.; and Wang, Y. (2019). A decisive content based image retrieval approach for feature fusion in visual and textual images. Knowledge-Based Systems, 179, 8-20, 2019. https://doi.org/10.1016/j.knosys.2019.05.001
[30] Xiaofeng; Qi; Lei; Zhang; Yao; Chen; Yong; Pi; Yi; and and, Q. (2018). Automated diagnosis of breast ultrasonography images using deep neural networks. Medical Image Analysis, 52, 185-198, 2018. https://doi.org/10.1016/j.media.2018.12.006
[31] Z., L.; M., P.; and S.Z., L. (2014). Learning discriminant face descriptor for face recognition. IEEE Transactions on Pattern Analysis and Machine Intelligence, 36(2), 289-302, 2014. https://doi.org/10.1109/TPAMI.2013.112
[32] Zhang, C.; Mei, Z.; Wu, B.; Yu, J.; andWang, Q. (2020). Query with assumptions for probabilistic relational databases. Tehnicki vjesnik, 27(3), 923-932, 2020. https://doi.org/10.17559/TV-20191123110408
[33] Zhang, S.; Yang, L.T.; Feng, J.; Wei, W.; Cui, Z.; Xie, X.; and Yan, P. (2021). A tensor-networkbased big data fusion framework for cyber-physical-social systems (cpss). Information Fusion, (76), 337-354, 2021. https://doi.org/10.1016/j.inffus.2021.05.014
[34] Zhou, Y.; Wang, Y.; and Wang, X.H. (2018). Face recognition algorithm based on wavelet transform and local linear embedding. Cluster Computing, 22, 1529-1540, 2018. https://doi.org/10.1007/s10586-018-2157-4
Additional Files
Published
2022-02-08
Issue
Section
Articles
License
ONLINE OPEN ACCES: Acces to full text of each article and each issue are allowed for free in respect of Attribution-NonCommercial 4.0 International (CC BY-NC 4.0.
You are free to:
-Share: copy and redistribute the material in any medium or format;
-Adapt: remix, transform, and build upon the material.
The licensor cannot revoke these freedoms as long as you follow the license terms.
DISCLAIMER: The author(s) of each article appearing in International Journal of Computers Communications & Control is/are solely responsible for the content thereof; the publication of an article shall not constitute or be deemed to constitute any representation by the Editors or Agora University Press that the data presented therein are original, correct or sufficient to support the conclusions reached or that the experiment design or methodology is adequate.
