SatQuery AI - An Interactive Vision-Language Assistant for Multimodal Remote Sensing Image Analysis through Text Queries
“SatQuery AI is an intelligent assistant that lets users chat with satellite imagery using plain English. Instead of requiring users to know complex GIS software or satellite data formats, the system automatically routes user queries to specialized AI models capable of analyzing single images, detecting changes over time, or combining radar and optical scans.”
Deconstructed Problem Blueprint
WHAT TO BUILD?
An agentic vision-language software platform with a web GUI that accepts single, multi-temporal, and cross-modal remote sensing images and processes natural language queries using orchestrated AI specialist models.
WHO IS IT FOR?
ISRO/SAC scientists, disaster management authorities, agricultural planners, urban developers, and non-expert government officials who need fast insights from satellite data.
WHY DOES IT EXIST?
Traditional remote sensing analysis requires specialized GIS expertise, manual tool selection, and disjointed single-task pipelines. Critical information is often spread across multi-sensor or multi-temporal formats that general-purpose AI and non-experts cannot easily interpret together.
INPUT → PROCESS → OUTPUT ARCHITECTURE PIPELINE
GeoTIFF/TIFF single satellite images, co-registered optical-SAR pairs, bi-temporal image pairs, and natural language text queries.
Agentic orchestration validates inputs, classifies the user query, routes it to domain-adapted vision-language models (fine-tuned on BigEarthNet/VRSBench/CDVQA), fuses optical and SAR data, and performs task-specific inference.
Evidence-grounded text answers, spatial change maps, highlighted bounding boxes, confidence scores, and downloadable audit reports.
CONSTRAINTS
- Must handle heavy geospatial raster files (GeoTIFFs) efficiently in a web environment
- Must use domain-adapted models fine-tuned on remote sensing datasets rather than relying on generic off-the-shelf VLMs
- Must provide an auditable execution trace including selected task, tools, and parameters
JURY METRICS
- Accuracy on benchmark tasks (VQA, captioning, grounding, change detection)
- Correctness of agentic tool routing and orchestration
- Inference latency and responsiveness of the web GUI
- Robustness against multi-sensor (optical vs. SAR) variations
CORE BOTTLENECK
Building a reliable agentic orchestrator that correctly translates arbitrary natural language queries into the exact combination of specialist computer vision and remote sensing tools without hallucinating parameters.
Why You Should Choose This PS (Pros)
- High national strategic importance directly backed by ISRO
- Clear, well-defined benchmarking pipeline (BigEarthNet, VRSBench, CDVQA)
- Tremendous 'wow factor' through agentic visual grounding and multi-image comparison
Challenges & Risks to Watch (Cons)
- Heavy deep learning models require significant GPU compute resources during development
- Handling large multi-gigabyte GeoTIFF rasters in web browsers can cause memory crashes if not optimized
- Co-registration alignment issues between multi-sensor or multi-temporal pairs
Recommended Tech Stack & Grand Finale Features
Killer Features to Impress the SIH Jury
View Official SIH 2026 Portal Raw Text▼
Background Remote-sensing imagery is widely used for agricultural monitoring, disaster management, urban planning, forest monitoring, water-resource assessment, infrastructure mapping, and environmental analysis. However, most existing remote-sensing AI solutions are developed as isolated applications for a single predefined task, such as land-cover classification, object detection, visual question answering, or change detection. These systems often require users to understand satellite-data characteristics, GIS workflows, model selection, and task-specific parameters. Consequently, non-expert users may find it difficult to obtain meaningful information from satellite imagery through simple natural-language queries.
Many operational remote-sensing questions cannot always be answered reliably using a single optical image. Relevant information may be distributed across paired or multiple observations acquired at different times or by different sensors. Optical and multispectral imagery provides spectral and contextual information, whereas synthetic aperture radar (SAR) provides complementary structural information and supports day-and-night acquisition through cloud cover. Multitemporal image pairs are required to identify and interpret changes over time, while co-registered optical–SAR pairs can provide more complete and reliable information than either modality alone.
A general-purpose large language model (LLM) or vision-language model (VLM) cannot be expected to perform these specialised tasks reliably without adaptation to remote-sensing imagery, sensor characteristics, and domain-specific terminology. The proposed solution must therefore include remote-sensing fine-tuning or domain adaptation and may employ multiple specialised models for different tasks. BigEarthNet.txt will serve as the primary dataset for adapting image–text representations to multisensor remote-sensing data. VRSBench and RSVQA will be used to evaluate single-image captioning, grounding, and visual question answering, while CDVQA will be used to evaluate multitemporal change-based visual question answering.
The novelty of SatQuery AI lies in its agentic, query-driven framework. Instead of applying a single generic VLM, the system selects and executes suitable remote-sensing specialist models, validates inputs, combines their outputs, and returns an evidence-grounded response.
Description The objective is to develop SatQuery AI, a software-based agentic vision-language assistant for analysing single and paired remote-sensing images through natural-language queries. Single-image understanding is a mandatory baseline, while the principal focus is joint reasoning over paired cross-modal and multitemporal imagery.
Defined Input Scope
Mandatory Functional Scope
Representative Queries
Agentic Model and Tool Orchestration The system may use multiple specialised components, such as a remote-sensing VQA or captioning model, a grounding model, a change-understanding or change-VQA model, and an optical–SAR fusion or information-extraction model.
The controller may perform internal task planning; however, only the observable execution trace, including the selected task, models or tools, permitted parameters, and outputs will be evaluated. Internal reasoning text is neither required nor evaluated.
Expected Solution The expected solution is an interactive GUI or web application with an agentic remote-sensing AI backend. It should accept supported image inputs and natural-language queries, select the appropriate specialist workflow, and return evidence-grounded textual and visual results.
The solution should include:
Each solution must demonstrate single-image VQA, one additional single-image task, multitemporal change understanding, optical–SAR paired-image analysis, and agentic model/tool orchestration. A generic LLM or VLM without remote-sensing adaptation will not satisfy the requirements.
Deliverables An interactive GUI or web application with an agentic remote-sensing AI backend, Codes and models including test and demonstration.
Implementation Scope The system shall support single optical/multispectral or SAR images, co-registered optical–SAR pairs, and bi-temporal pairs in GeoTIFF/TIFF or approved benchmark formats. It must perform single-image VQA, one additional single-image task, change analysis, optical–SAR joint analysis, and agentic model/tool selection through an interactive GUI or web application.
Evaluation/Judging Criteria Final evaluation will use prescribed public benchmark test subsets and an ISRO/SAC evaluation dataset. Scores will be normalised before combining different metrics.
Add 'Evaluation/Judging Criteria' table here Public benchmarks will be evaluated using the prescribed test splits. The ISRO/SAC evaluation set will contain pre-georeferenced and co-registered Cartosat-2S optical and RISAT SAR image pairs, with task-specific reference answers, labels, bounding boxes, or masks, as applicable. Evaluation annotations will not be disclosed to participating teams.