Overview of the SISAP 2025 Indexing Challenge.
- Eric Sadit Tellez,
- Edgar Chávez,
- ,
- Vladimir Mic
- SECIHTI,
- Ensenada Center for Scientific Research and Higher Education,
- ,
- ,
- ,
- Aarhus University
Research Output:
Conference Article in Proceeding or Book/Report chapter
Article in proceedings
Peer-reviewOpen access
Publication Information
Output type
Research Output:
Conference Article in Proceeding or Book/Report chapter
Article in proceedings
Peer-reviewOriginal language
EnglishPages from-to (Number of pages)
Pages 403-414 (12 pages)Publication milestones
- Published - 2025
Publication status
Published - 2025
Publisher
Springer, United States, GermanyISBN (Print)
978-3-032-06068-6ISBN (Electronic)
978-3-032-06069-3Host publication title
SISAPAbstract
This paper summarizes the innovative solutions presented at the third edition of the SISAP Indexing Challenge held at SISAP 2025.
The challenge featured two distinct tasks involving vector embeddings derived from a large corpus using neural encoders. It proposed the following two tasks under strict memory and computational constraints:
– Task 1: Approximate nearest neighbor search achieving an average
recall of at least 0.7 for 30-NN, using out-of-distribution objects as
queries.
– Task 2: k-NN (k = 15) graph construction for large datasets, requiring an average recall of at least 0.8.
Both tasks required solutions to operate within strict resource limits: 16 GB of RAM, 8 virtual CPUs, and a 12-hour wall-clock time for the end-to-end pipeline (including data loading, pre-processing, indexing, and searching). Each task imposes different minimum quality requirements and ranking specifications. Participants developed strategies such as data compression, optimized indexing, and efficient search algorithms to meet these constraints. This paper details the challenge design, explains the evaluation framework, and provides an overview of the submitted solutions.
The challenge featured two distinct tasks involving vector embeddings derived from a large corpus using neural encoders. It proposed the following two tasks under strict memory and computational constraints:
– Task 1: Approximate nearest neighbor search achieving an average
recall of at least 0.7 for 30-NN, using out-of-distribution objects as
queries.
– Task 2: k-NN (k = 15) graph construction for large datasets, requiring an average recall of at least 0.8.
Both tasks required solutions to operate within strict resource limits: 16 GB of RAM, 8 virtual CPUs, and a 12-hour wall-clock time for the end-to-end pipeline (including data loading, pre-processing, indexing, and searching). Each task imposes different minimum quality requirements and ranking specifications. Participants developed strategies such as data compression, optimized indexing, and efficient search algorithms to meet these constraints. This paper details the challenge design, explains the evaluation framework, and provides an overview of the submitted solutions.
Access to documents
Accepted author manuscript, 421.92 KB
Related Event
Title
International Conference on Similarity Search and Applications
Event type
ConferenceDegree of recognition
International eventDate
01/10/2025 - 03/10/2025Location
ReykjavikIceland
