Check out this pull request on 

Review Jupyter notebook visual diffs & provide feedback on notebooks.

Powered by ReviewNB

ArcGIS/geosaurus/issues/4245

View / edit / reply to this conversation on ReviewNB

moonlanderr commented on 2020-09-03T07:04:32Z
----------------------------------------------------------------

Create geometry of AOI

Generate water body mask

Create water body mask

View / edit / reply to this conversation on ReviewNB

moonlanderr commented on 2020-09-03T07:04:33Z
----------------------------------------------------------------

Turbidity represents the level of suspended sediments in water also indicating water clarity or how clear is the water. It is mainly caused by the presence of silt, algae in a water body, or industrial waste disposed in the rivers by mining activity, factory operation, logging, etc.

Traditionally, turbidity is analyzed by evaluating water samples taken during field measurements. However, field studies are expensive, time and labor intensive, besides, during lockdown field surveys cannot be undertaken. Thus, a good alternative to field survey measurements is satellite remote sensing data, which can as well capture both spatial and temporal variations in river turbidity levels. Accordingly, Sentinel-2 multispectral data is used in the current study to evaluate the changes in river turbidity during COVID-19 lockdown, near the holy city of Allahabad, India.

The case study area of Allahabad is in the northern part of India, at the confluence of Ganga and Yamuna river, which is considered as one of the important cities in Hindu religion. Everyday thousands of Hindu devotees visit the city and disposes waste directly in the rivers. The small-scale factories situated in the city also disposes its waste into the rivers adding to the water pollution. On 25th March 2020, lockdown was announced in India for controlling the COVID-19 spread which meant total shutdown of industries and restricted human movement. The lockdown resulted in reduction of turbidity hence improved water quality in rivers throughout the country. This notebook will elaborate the steps to measure this change in turbidity using arcgis api for python tools.

View / edit / reply to this conversation on ReviewNB

moonlanderr commented on 2020-09-03T07:04:34Z
----------------------------------------------------------------

Necessary

View / edit / reply to this conversation on ReviewNB

moonlanderr commented on 2020-09-03T07:04:34Z
----------------------------------------------------------------

1. check if this line is needed

from arcgis import *

2. the arcgis imports can be grouped together like this,

import pandas as pd
from datetime import datetime
from ipywidgets import HBox, VBox, Label, Layout
import arcgis

from arcgis import *

from arcgis.features import GeoAccessor, GeoSeriesAccessor

from arcgis.raster.analytics import convert_feature_to_raster, convert_raster_to_feature

from arcgis.raster.functions import extract_band, greater_than, clip, remap, colormap, stretch

from arcgis.features.analysis import dissolve_boundaries

View / edit / reply to this conversation on ReviewNB

moonlanderr commented on 2020-09-03T07:04:35Z
----------------------------------------------------------------

Sentinel-2 Views was used in the analysis: - this multispectral and multitemporal imagery consists of 13 bands with 10, 20, and 60m spatial resolution, which is rendered on-the-fly and available for visualization and analytics. This imagery layer pulls directly from the Sentinel-2 on AWS collection and is updated daily with new imagery. 

View / edit / reply to this conversation on ReviewNB

moonlanderr commented on 2020-09-03T07:04:36Z
----------------------------------------------------------------

add some description what is being done in this step of searching allahabad aoi, also why it is needed

View / edit / reply to this conversation on ReviewNB

moonlanderr commented on 2020-09-03T07:04:37Z
----------------------------------------------------------------

describe the methodology briefly in words as elaborated in the graphics

View / edit / reply to this conversation on ReviewNB

moonlanderr commented on 2020-09-03T07:04:38Z
----------------------------------------------------------------

Sentinel-2 Views imagery layers consists data for the whole world and span different time periods. Thus the first step is to filter out the data of the river in the Allahabad region prior to lockdwon and during the period of lockdown.

View / edit / reply to this conversation on ReviewNB

moonlanderr commented on 2020-09-03T07:04:39Z
----------------------------------------------------------------

Create geometry of area of interest (AOI)

View / edit / reply to this conversation on ReviewNB

moonlanderr commented on 2020-09-03T07:04:40Z
----------------------------------------------------------------

The geometry of AOI was created for filtering out the Sentinel-2 tiles for the study area.

View / edit / reply to this conversation on ReviewNB

moonlanderr commented on 2020-09-03T07:04:41Z
----------------------------------------------------------------

briefly refer where are you getting the fid=1

View / edit / reply to this conversation on ReviewNB

moonlanderr commented on 2020-09-03T07:04:42Z
----------------------------------------------------------------

describe briefly what is being done here

View / edit / reply to this conversation on ReviewNB

moonlanderr commented on 2020-09-03T07:04:43Z
----------------------------------------------------------------

briefly refer how you are getting category=1

View / edit / reply to this conversation on ReviewNB

moonlanderr commented on 2020-09-03T07:04:43Z
----------------------------------------------------------------

briefly refer where you are getting the object id for filtering

View / edit / reply to this conversation on ReviewNB

moonlanderr commented on 2020-09-03T07:04:45Z
----------------------------------------------------------------

provide a brief what will be done here along with the filter_by parameters source reference

View / edit / reply to this conversation on ReviewNB

moonlanderr commented on 2020-09-03T07:04:45Z
----------------------------------------------------------------

Generate water body mask

View / edit / reply to this conversation on ReviewNB

moonlanderr commented on 2020-09-03T07:04:46Z
----------------------------------------------------------------

Create normalized difference water index (NDWI) raster

View / edit / reply to this conversation on ReviewNB

moonlanderr commented on 2020-09-03T07:04:47Z
----------------------------------------------------------------

Normalized difference water index (NDWI) is a satellite based index used for mapping and detecting the surface water bodies. Water absorbs electromagnetic radiation in visible to infrared spectrum, that is why Green and Near Infrared bands are used to detect the water bodies. In the current study, band 3 (green) and band 8 (NIR) of Sentinel-2 is used for generating NDWI raster. Accordingly the bands are first extracted followed by creation of the NDWI raster as follows:

move the Extract bands section within this section of -Generate water bodies mask- after this paragraph so that it becomes clear why these particular bands are extracted, and change the content accordingly

View / edit / reply to this conversation on ReviewNB

moonlanderr commented on 2020-09-03T07:04:48Z
----------------------------------------------------------------

Binary raster is created from NDWI raster using a threshold value. The binary raster consists of two classes of water and non-water pixels where pixels with value greater than 0.03 are considered as water. Accordingly this threshold value of 0.03 is used for creating the binary raster using the greater_than function.

View / edit / reply to this conversation on ReviewNB

moonlanderr commented on 2020-09-03T07:04:49Z
----------------------------------------------------------------

Create water body mask

add text explaining this step

View / edit / reply to this conversation on ReviewNB

moonlanderr commented on 2020-09-03T07:04:50Z
----------------------------------------------------------------

The binary rasters are converted to feature layer for extracting the boundaries of the water bodies.

View / edit / reply to this conversation on ReviewNB

moonlanderr commented on 2020-09-03T07:04:50Z
----------------------------------------------------------------

In the feature layer returned above, 'gridcode=0' represents non water class and 'gridcode=1' represents water class. Thus the water polygons with 'gridcode=1 are selected using the query function from the dataframe of the feature layer and saved into a new feature layer created using gis.content.import function consisting of these water polygons.

View / edit / reply to this conversation on ReviewNB

moonlanderr commented on 2020-09-03T07:04:51Z
----------------------------------------------------------------

explain the parameter t

View / edit / reply to this conversation on ReviewNB

moonlanderr commented on 2020-09-03T07:04:52Z
----------------------------------------------------------------

Finally the features in the layer water_poly are dissolved using dissolve_boundaries function on the basis of gridcode column to obtain the water body mask in the following.

View / edit / reply to this conversation on ReviewNB

moonlanderr commented on 2020-09-03T07:04:53Z
----------------------------------------------------------------

The Normalize Difference Turbidity Index (NDTI) which is estimated using the spectral reflectance values of the water pixels is used to estimate the turbidity in water bodies. It uses the phenomenon that the electromagnetic reflectance is higher in green spectrum than the red spectrum for clear water. Hence, with increase in turbidity the reflectance of red spectrum also increases. Accordingly in the current study Sentinel-2 green (band 3) and red (band 4) bands are used to create the NDTI raster in the following.

View / edit / reply to this conversation on ReviewNB

moonlanderr commented on 2020-09-03T07:04:54Z
----------------------------------------------------------------

Once the NDTI raster is prepared, its portion for the water body is extracted using the water body mask obtained before. To do that first the geometry of dissolve_f is created as follows:

dissolve_f -is it the water mask created before? diss_f - is this one?

View / edit / reply to this conversation on ReviewNB

moonlanderr commented on 2020-09-03T07:04:55Z
----------------------------------------------------------------

Third step is to apply colormap for visualizing the results.

View / edit / reply to this conversation on ReviewNB

moonlanderr commented on 2020-09-03T07:04:56Z
----------------------------------------------------------------

Finally all the above mentioned functions are used inside a loop and applied on the satellite imagery of the river from two different time periods: one before lockdown and the second, during the lockdown period, to visualize and compare the difference in the turbidity levels in them.

View / edit / reply to this conversation on ReviewNB

moonlanderr commented on 2020-09-03T07:04:57Z
----------------------------------------------------------------

Result Visualization

View / edit / reply to this conversation on ReviewNB

moonlanderr commented on 2020-09-03T07:04:57Z
----------------------------------------------------------------

The NDTI rasters and their corresponding Sentinel-2 tiles are now accessed from the portal and used for visualization.

View / edit / reply to this conversation on ReviewNB

moonlanderr commented on 2020-09-03T07:04:58Z
----------------------------------------------------------------

4 map wigets are created representing the NDTI raster and corresponding Sentinel-2 for 9th March (before lockdown) and 13th April (during lockdown).

View / edit / reply to this conversation on ReviewNB

moonlanderr commented on 2020-09-03T07:04:59Z
----------------------------------------------------------------

All the maps are synchronized with each other using MapView.sync_navigation functionality. It helps in comparing the river turbidity before and during lockdown. Detailed description about advanced map widget options can be referred here

View / edit / reply to this conversation on ReviewNB

moonlanderr commented on 2020-09-03T07:05:00Z
----------------------------------------------------------------

if possible add a legend for the pixel colors

if possible point out which one is the Yamuna & Ganga river

View / edit / reply to this conversation on ReviewNB

moonlanderr commented on 2020-09-03T07:05:01Z
----------------------------------------------------------------

The maps above shows the spatio-temporal variation in the Ganga and Yamuna river turbidity due to Covid-19 lockdown. The satellite imageries used in the analysis are from before lockdown situation, dated 9th march 2020 (bottom-left), and during the lockdown situation dated 13th April 2020 (botton-right).

It can be seen that turbidity declined in both the Ganga and Yamuna river during lockdown indicated by the Red pixels representing high turbidity, turning to Orange, Yellow and finally Green, which is the lowest level of turbidity. Compared to the Yamuna the Ganga river had a significant impact with the whole river stretch turning from red and yellow to mostly green pixels. The turbidity at the Sangam or the confluence point also declined followed by the same in the stretch downstream.

View / edit / reply to this conversation on ReviewNB

moonlanderr commented on 2020-09-03T07:05:02Z
----------------------------------------------------------------

Ganga and Yamuna river falls under the world's most polluted river due to dumping of waste from industrial and religious activities indicated by its highly turbid water. During the period of lock down there was a significant improvement in the pollution level of both the rivers due to complete shutdown of the above-mentioned functions. This change was investigated using Sentinel-2 satellite imageries of the river stretches, before and during the lockdown periods, with results suggesting substantial decrease in turbidity levels thereby validating the same. The same methodology can be implemented to study the changes in river turbidity for other regions with Sentinel-2 data using ArcGIS platform.