A Novel Fuzzy ARTMAP Architecture with Adaptive FeatureWeights based on Onicescu’s Informational Energy
Keywords:
Fuzzy ARTMAP, feature weighting, LVQ, Onicescu’s informational energyAbstract
Fuzzy ARTMAP with Relevance factor (FAMR) is a Fuzzy ARTMAP (FAM) neural architecture with the following property: Each training pair has a relevance factor assigned to it, proportional to the importance of that pair during the learning phase. Using a relevance factor adds more flexibility to the training phase, allowing ranking of sample pairs according to the confidence we have in the information source or in the pattern itself.
We introduce a novel FAMR architecture: FAMR with Feature Weighting (FAMRFW). In the first stage, the training data features are weighted. In our experiments, we use a feature weighting method based on Onicescu’s informational energy (IE). In the second stage, the obtained weights are used to improve FAMRFW training. The effect of this approach is that category dimensions in the direction of relevant features are decreased, whereas category dimensions in the direction of non-relevant feature are increased. Experimental results, performed on several benchmarks, show that feature weighting can improve the classification performance of the general FAMR algorithm.
References
R. Andonie and A. Ca¸taron. Feature ranking using supervised neural gas and informational energy. In Proceedings of IEEE International Joint Conference on Neural Networks (IJCNN2005), Canada, Montreal, July 31 - August 4, 2005. http://dx.doi.org/10.1109/ijcnn.2005.1556036
R. Andonie, A. Ca¸taron, and L. Sasu. Fuzzy ARTMAP with feature weighting. In Proceedings of the IASTED International Conference on Artificial Intelligence and Applications (AIA 2008), Innsbruck, Austria, Febr. 11-13, 2008, 91-96.
R. Andonie and F. Petrescu. Interacting systems and informational energy. Foundation of Control Engineering, 11, 1986, 53-59.
R. Andonie and L. Sasu. Fuzzy ARTMAP with input relevances. IEEE Transactions on Neural Networks, 17, 2006, 929-941. http://dx.doi.org/10.1109/TNN.2006.875988
A. Asuncion and D. J. Newman. UCI machine learning repository, 2007. University of California, Irvine, School of Information and Computer Sciences http://www.ics.uci.edu/»mlearn/MLRepository.html
I. Dzi¸tac and B. E. B˘arbat. Artificial intelligence + distributed systems = agents. International Journal Computers, Communications, and Control, 4, 2009, 17-26. http://dx.doi.org/10.15837/ijccc.2009.1.2410
G. A. Carpenter, S. Grossberg, N. Markuzon, J. H. Reynolds, and D. B. Rosen. Fuzzy ARTMAP: A Neural Network Architecture for Incremental Supervised Learning of Analog Multidimensional Maps. IEEE Transactions on Neural Networks, 3, 1992, 698-713. http://dx.doi.org/10.1109/72.159059
G. A. Carpenter, B. L. Milenova, and B. W. Noeske. Distributed ARTMAP: A neural network for fast distributed supervised learning. Neural Networks, 11, 1998, 793-813. http://dx.doi.org/10.1016/S0893-6080(98)00019-7
G. A. Carpenter and W. Ross. ART-EMAP: A neural network architecture for learning and prediction by evidence accumulation. IEEE Transactions on Neural Networks, 6, 1995, 805-818. http://dx.doi.org/10.1109/72.392245
D. Charalampidis, G. Anagnostopoulos, M. Georgiopoulos, and T. Kasparis. Fuzzy ART and Fuzzy ARTMAP with adaptively weighted distances. In Proceedings of the SPIE, Applications and Science of Computational Intelligence, Aerosense, 2002. http://dx.doi.org/10.1117/12.458723
I. Dagher, M. Georgiopoulos, G. L. Heileman, and G. Bebis. An ordering algorithm for pattern presentation in Fuzzy ARTMAP that tends to improve generalization performance. IEEE Transactions on Neural Networks, 10, 1999, 768-778. http://dx.doi.org/10.1109/72.774217
I. Dagher, M. Georgiopoulos, G. L. Heileman, and G. Bebis. Fuzzy ARTVar: An improved fuzzy ARTMAP algorithm. In Proceedings IEEE World Congress Computational Intelligence WCCI'98, Anchorage, 1998, 1688-1693. http://dx.doi.org/10.1109/ijcnn.1998.687110
J. C. Principe et al. Information-theoretic learning. In S. Haykin, editor, In Unsupervised Adaptive Filtering. Wiley, New York, 2000.
E. Gomez-Sanchez, Y. A. Dimitriadis, J. M. Cano-Izquierdo, and J. Lopez-Coronado. ¹ARTMAP: Use of mutual information for category reduction in fuzzy ARTMAP. IEEE Transactions on Neural Networks, 13, 2002, 58-69. http://dx.doi.org/10.1109/72.977271
S. Guia¸su. Information theory with applications. McGraw Hill, New York, 1977.
B. Hammer, D. Schunk, T. Bojer, and T. K. von Toschanowitz. Relevance determination in learning vector quantization. In Proceedings of the European Symposium on Artificial Neural Networks (ESANN 2001), Bruges, Belgium, 2001, 271-276.
B. Hammer, M. Strickert, and T. Villmann. Supervised neural gas with general similarity measure. Neural Processing Letters, 21, 2005, 21-44. http://dx.doi.org/10.1007/s11063-004-3255-2
B. Hammer and T. Villmann. Generalized relevance learning vector quantization. Neural Networks, 15, 2002, 1059-1068. http://dx.doi.org/10.1016/S0893-6080(02)00079-5
C. P. Lim and R. Harrison. ART-Based Autonomous Learning Systems: Part I - Architectures and Algorithms. In L. C. Jain, B. Lazzerini, and U. Halici, editors, Innovations in ART Neural Networks. Springer, 2000. http://dx.doi.org/10.1007/978-3-7908-1857-4_6
C. P. Lim and R. F. Harrison. An incremental adaptive network for on-line supervised learning and probability estimation. Neural Networks, 10, 1997, 925-939. http://dx.doi.org/10.1016/S0893-6080(96)00123-2
S. Marriott and R. F. Harrison. A modified fuzzy ARTMAP architecture for the approximation of noisy mappings. Neural Networks, 8, 1995, 619-641. http://dx.doi.org/10.1016/0893-6080(94)00110-8
T. M. Martinetz, S. G. Berkovich, and K. J. Schulten. Neural-gas network for vector quantization and its application to time-series prediction. IEEE Transactions on Neural Networks, 4, 1993, 558-569. http://dx.doi.org/10.1109/72.238311
S. Min-Kyu, J. Murata, and K. Hirasawa. Function approximation using LVQ and fuzzy sets. In Proceedings of the IEEE International Conference on Systems, Man, and Cybernetics, Tucson, AZ, 2001, 1442-1447.
O. Onicescu. Theorie de l'information. Energie informationnelle. C. R. Acad. Sci. Paris, Ser. A-B, 263, 1966, 841—842.
O. Parsons and G. A. Carpenter. ARTMAP neural networks for information fusion and data mining: map production and target recognition methodologies. Neural Networks, 16, 2003, 1075-1089. http://dx.doi.org/10.1016/S0893-6080(03)00007-8
M. Taghi, V. Baghmisheh, and P. Nikola. A Fast Simplified Fuzzy ARTMAP Network. Neural Processing Letters, 17, 2003, 273-316. http://dx.doi.org/10.1023/A:1026004816362
S. J. Verzi, G. L. Heileman, M. Georgiopoulos, and M. J. Healy. Boosted ARTMAP. In Proceedings IEEE World Congress Computational Intelligence WCCI'98, 1998, 396-400. http://dx.doi.org/10.1109/ijcnn.1998.682299
J. Williamson. Gaussian ARTMAP: A neural network for fast incremental learning of noisy multidimensional maps. Neural Networks, 9, 1996, 881-897. http://dx.doi.org/10.1016/0893-6080(95)00115-8
Published
Issue
Section
License
ONLINE OPEN ACCES: Acces to full text of each article and each issue are allowed for free in respect of Attribution-NonCommercial 4.0 International (CC BY-NC 4.0.
You are free to:
-Share: copy and redistribute the material in any medium or format;
-Adapt: remix, transform, and build upon the material.
The licensor cannot revoke these freedoms as long as you follow the license terms.
DISCLAIMER: The author(s) of each article appearing in International Journal of Computers Communications & Control is/are solely responsible for the content thereof; the publication of an article shall not constitute or be deemed to constitute any representation by the Editors or Agora University Press that the data presented therein are original, correct or sufficient to support the conclusions reached or that the experiment design or methodology is adequate.
