Studies on genetic diversity, population structure, and cross-transferability have chosen the simple sequence repeat (SSR) marker as their preferred marker. This study examined the genetic diversity and population structure of 22 Capsicum spp. collection accessions (C. annuum, C. chinense and C. frutescens) using the recently developed 20 SSRs derived from the genus Capsicum. The fruit’s phenotypic traits, such as shape, placenta type, apex, pericarp thickness, and colour at the immature and mature stages of development were investigated. In 22 accessions, 13 polymorphic SSRs were used for analysis. The total number of alleles (54) (ranging from 2 to 7, with an average of 4.15 per locus) as well as polymorphism information content values (0.34 to 0.86, with a mean of 0.56). Using SSRs, neighbour-joining and factorial analyses of 22 Capsicum accessions formed three major clusters, primarily based on their distinct types, and having no relationship with their geographical locations. Model-based STRUCTURE analysis also showed three genetically distinct populations (K = 3). C. frutescens accessions (chilies) were clustered together, and C. annuum accessions were separated into two clusters, indicating substantial genetic variation in the collection. Moreover, some admixture exists in all three clusters. The mature fruit colour, shape, and size vary between accessions. These thirteen SSR markers discriminate between the three Capsicum species in the collection. They may be implemented to conduct genetic management and marker-assisted breeding in Capsicum crops.