The devastating tsunami of 2004 in the Indian Ocean goes into history as one of the worst tragedies in recent times. It killed close to 2 lakh people and rendered 17 lakh homeless, across 14 countries. During the massive scale of rescue and relief operations that followed, satellite images that captured details of the affected areas, including the most remote islands were of great help. However, manually searching through these images to find the closest hospital or a safe building to stay in can be tedious and time-consuming, slowing down the rescue work. In a new study, researchers from the Indian Institute of Technology Bombay have developed a better method to search satellite images with commonly-used search phrases in English.
Today, India has about 23 remote sensing satellites that continuously collect data and add to a massive database, which is used in the time of crisis. The images captured here contain low-level features like colour, texture and shape, which are understood by the system, but hardly make sense for the user. For example, a user may be interested in a search for ‘flooded residential areas’. Since the images capture the flooded areas with a different colour or texture, the system fails to understand the meaning of it and hence the user is forced to form a query like ‘show me all the grey coloured regions’. This gap in understanding is called the semantic gap.
The researchers of this study have developed a ‘semantic’ framework that one can query and receive information about a region based on its spatial and directional relationship with surrounding areas. Semantics refers to meaning or connection of meanings of a set of signs. Their approach links the high-level thinking of the user with the low-level representation of the images, which could speed up the analysis time.
“A semantics-enabled framework facilitates understanding about a domain, like how humans process information, and converts it into knowledge”, says Prof. Surya Durbha from IIT Bombay, who is also an author of the study published in the IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing.
The new framework provides information about the contents of an image like identifying a building, farmland or empty land, along with its spatial and directional relationship with its surroundings. In a post-flood disaster scenario, such information can be used to find all buildings that are surrounded by water and has a road that is unaffected or partially flooded. The researchers have used a specific computer language that can represent this knowledge and have developed a domain-specific database of concepts that make the implicit knowledge explicit.
The framework has two components—an offline module and an online module. The offline module extracts the features like colour, texture and shape from satellite images. The various spatial and directional relationships between regions are also encoded and later retrieved while searching using a reasoning engine. The online mode uses a graphical user interface to facilitate selecting different land use and land cover categories such as a river, cropland, partially-flooded cropland and so on. Users can also choose the required spatial configurations in the images and search them to retrieve relevant images.
“Our system can not only rapidly identify the affected areas but also identify the areas based on their spatial and directional relationships. This enables a rapid search of the contents of huge archives of remote sensing imagery based on specific needs”, says Prof. Durbha, highlighting the effectiveness of the framework.
Is the use of the framework limited to rescue operations? No, say the researchers, pointing out that it can be used by anyone to find images that satisfy their purpose. The users can install it on any computer and specify the location of the images. Post this, the framework processes the data and makes it ready for querying. Also, the system can be customised to handle different disaster situations.
The study is a step in the right direction when the world is facing a rise in the number of natural calamities. The developed framework can help to evolve appropriate management strategies, enabling the agencies to rapidly process post-disaster data and facilitate quick responses that could save many lives.