CAREER: Aligning Image Retrieval Systems with Human Notions of Similarity

NSF Award Search · 01002829DB NSF RESEARCH & RELATED ACTIVIT · $599,998 · view on nsf.gov ↗

Abstract

Fine-grained visual categorization involves identifying subtle differences between highly similar visual categories, such as distinguishing between two closely related bird species or recognizing different models of a car. While these capabilities are critical in a variety of fields, including biodiversity research, forensic investigations, and e-commerce, the task is challenging because differences between categories can be small, while variations within a category can be large. For example, two different bird species may look very similar, while male and female birds of the same species may look very different. Visual categorization in fine-grained domains is often treated as an image retrieval problem, where the label for a query image is determined based on the labels of the most visually similar images. An image retrieval approach typically performs better than standard classification approaches on fine-grained visual categorization tasks, especially in domains with very large numbers of classes. However, image-retrieval approaches often fail because a retrieved image that is visually similar is not necessarily from the same class, and potential images from the same class may not be retrieved due to low visual similarity. Moreover, the features learned by standard image retrieval models are often biased towards overall visual similarity rather than task-specific or domain-specific notions of importance. This limitation can hinder analysts and domain experts who may want

Key facts

NSF award ID
2441774
Awardee
Saint Louis University (MO)
SAM.gov UEI
JNBLLTBTLLD8
PI
Abigail Stylianou
Primary program
01002829DB NSF RESEARCH & RELATED ACTIVIT
All programs
CAREER-Faculty Erly Career Dev, INFO INTEGRATION & INFORMATICS, EXP PROG TO STIM COMP RES
Estimated total
$599,998
Funds obligated
$345,135
Transaction type
Continuing Grant
Period
06/15/2025 → 05/31/2030