An Investigation into the Relationship between Curse of Dimensionality and Dunning-Kruger Effect

Abstract

This study addresses a novel perspective for analyzing the source of confidence in human behavior. The concept of confidence was examined via the relationship between two phenomena in the area of machine learning and psychology, namely the Dunning-Kruger effect and the curse of dimensionality. A relationship was established between these two phenomena which were investigated in the light of neuroscience. This study claims that confidence is highly related with the total time it takes to reach specific information and this relationship is inversely proportional. Image gender classification algorithm was used to analyze this relationship for this study and the curves which were obtained as a result of this analysis was compared with the curve of Dunning-Kruger effect and curse of dimensionality. This relationship has been explained by the knowledge of human's problem-solving ability and mathematical models of memory.