SAR and ENVI
PH330 Lab Exercise:
Introduction to ENVI Software: Image Processing of
RADARSAT SAR Data
PH330 Students and Colleagues: Check Here for Lab Updates.
Note to Students:
The data are in the directory /local/lvos/classes/ph330sarlab/
Note that you cannot do email from this account.
http://www.geo.mtu.edu/great_lakes/icegroup/ph330/sarlab.html
Point of contact: Drew Pilant
(Please, don't hesitate to call or email me with questions and especially
with problems.)
anpilant@mtu.edu 487-2083
Office: Fisher 124 D
Geol. Eng. & Sciences system administrator: Mike Dolan geomaint@mtu.edu -2167
Introduction to ENVI: Image Processing of RADARSAT SAR Data
Abstract
A C-band RADARSAT synthetic aperture radar (SAR) ScanSAR image is studied using the ENVI image processing software. The image data file is symbolically linked to the analyst's home directory, the image dimensions are calculated, a header file is created, the image 970401_keweenaw.envi is displayed, a 400x400 pixel subset image is created, a contrast enhancement tool is applied, a JPEG image file is exported, the image is smoothed with a median filter to remove speckle, an unsupervised classification is performed and color lookup tables are applied. In performing these tasks, the analyst gains a functional familiarity with a sophisticated image processing toolkit. SAR imagery of the Keweenaw Peninsula on Lake Superior provides an introduction to the somewhat exotic interaction of 5 cm wavelength electromagnetic radiation with the landscape: ice, water, forest, topography, snow, anthropogenic features. The concepts of radar illumination, backscatter, surface roughness, topographic effects and dielectric constant are linked by image processing and interpretation.
Objectives of PH330 Lab Exercise 2 are to:
- introduce the ENVI software;
- introduce some image processing and interpretation concepts germane to synthetic aperture radar (SAR) data;
- become more familiar with some useful UNIX commands.
Main themes:
Beginning an ENVI session: first start IDL, then start ENVI
Login to a machine in Dow 708 (borealis) or 709 (australis).
From an xterm, type the following commands (you get an xterm by pressing
the right mouse button in the blue (open) part of the screen, go to X-Programs
> Xterm)
australis01(7)% idl
IDL> envi
After your ENVI session:
Be sure to "Quit" ENVI when done, as we need all the licenses.
General ENVI comments:
- When you first try to open a file in ENVI, it looks in this directory
(in the Filter box in the widget):
/local/rsi/rsi/envi/data/
Your data reside (symbolically) in
~/ph330sarlab/
so enter
~/ph330sarlab/*
in the "Filter" (top) box of the ENVI File > Open Image File widget
(the * instructs ENVI to list all the available files in that directory)
-ENVI will not allow you to proceed if it is waiting for a command in an open widget. For example, if you are working on your image, and decide to open another image, you cannot resume working on your first image until you either finish opening the second image or 'Cancel' the Open File widget.
- sometimes you have to hit return and "Ok" to make function go.
- if ENVI hangs (unlikely), try to terminate the "process" as described above.
- generally, ENVI responds to single mouse clicks (double clicks are used for opening files and other 'operating system' types of commands).
- ENVI hypertext help is under System >> ENVI Online Help.
It may be useful to start up another xterm for executing unix commands while the other australis01.ice xterm is locked up using ENVI (for example, to do an 'ls' to verify that the image files you are writing from ENVI are actually going into your home directory). To do this, type
australis01(60)% xterm -title SARlab &
[1] 9474
This starts another xterm, and runs it in the background (&) so that the original xterm remains available for accepting commands. (Some shells do not recognize this command. If so, you can open an xterm using the mousebutton options available in the background region of your desktop.
(These notes and commands assume that you are in your australis01.ice
home directory. If you are elsewhere, or I made a mistake, you may need
to verify your path location and reference files accordingly.) The unix
command
pwd
tells you where you are. The command
cd
with no arguments takes you to your home or working directory by default.
There are 10 image processing tasks.
Please write image files to your the ph330sarlab subdirectory that you will create in your home directory. You may submit your written material on hardcopy, or electronically in an ascii file (a basic text file, such as an email message) called "ph330sarlab_commentary.txt".
Task 1: Symbolically Link the SAR Image to Your Directory (5 pts)
The SAR image we will use is about 1 Mbyte in size (1030225 bytes) (about
an order of magnitude larger than Bill Gates first hard drive). We can
save a lot of space if we 'link' the files to our own directories, rather
than copying them. The Unix command is 'ln'. Once linked, you can use the
file as if it was in your directory, though it actually resides elsewhere.
Create a subdirectory in your home directory called 'ph330sarlab'.
australis01(0)% mkdir ph330sarlab
australis01(0)% cd ph330sarlab
or,
australis01(0)% cd !$ (shorthand version)
The Lab 1 data will be in the directory called ph330_sar_lab:australis01(0)%
ls
/local/lvos/classes/ph330_sar_lab/
970401_keweenaw.envi
(list the contents of ph330sarlab directory.)
australis01(0)% ls -l /local/lvos/classes/ph330_sar_lab/
-rw-r--r-- 1 ice 1030225 Oct 17 17:44 970401_keweenaw.envi
(list the contents and filesizes of ph330_sar_lab directory.)
Link the data to your directory:
australis01(2)% ln -s /local/lvos/classes/ph330_sar_lab/970401_keweenaw.envi .
If you do an ls in your directory, it shows the file with an @ symbol indicating that it is linked. Execute the "ls -l" command above to obtain the file size of the original data.
-----------------------------------------------------------
Task 2: Figure Out How Many Rows and Columns Are in the SAR Image (5 pts)
This is a common problem. Your buddy gives you some excellent imagery, but forgets to tell you the number of rows and columns in the image. You must be a detective because without a header on the data file, your software can't read the data. Your task is to determine the number of rows and columns in the image based on the file size. (This assumes that the image file contains only data, no header or trailer information.)
ENVI references the image file and header information separately. The image file is simply an n-dimensional array, consisting of x number of columns, y number of rows and n number of bands. It is in binary format. In this exercise, we are using only one band of data, typical of radar data, so the image file is a 1-dimensional array of y-rows and x-columns. The header file (filename.hdr) is an ascii file containing 'header' information such as y and x (row, column) dimensions, interleave format, etc. You will create a header file for the file of interest. ENVI will prompt you, but you need to know the x-y dimensions. You will compute this. Note that I didn't give you the x-y dimensions, but I do tell you that the file is square. But knowing that it is square, and that these are 8 bit data (as opposed to 16 bit or 10 bit or ??), you can do the math. Type
australis01(4)% ls -l /local/lvos/classes/ph330_sar_lab/970401_keweenaw.envi
-rw-r--r-- 1 ice 1030225 Oct 17 17:44 970401_keweenaw.envi
1030225 is the file size in bytes. Now what? Well, if you were very intuitive and extremely lucky, you might be able to factor this into its correct x-y row-column dimensions. Fortunately, we have more info. In this operating system (Sun-Solaris) (and in most o.s.), 1 byte= 8 bits. We know, or we guess, that this is 8-bit data (i.e., the digital number (DN) representing backscatter intensity of each pixel is an integer value between 0-255, that is, between 0 and 2^^8). Therefore, each byte in this file is one datum, or one pixel. (16 bit data would be 2 bytes each.) Therefore this image consists of 1030225 pixels, in some unknown combination of rows and columns. Now all we have to do is find the row-column factors that multiply to yield a product of 1030225 bytes. Hint: the image is square. Take the square root of the total file size in bytes; the result is the number of pixels in each row and column. (Type 'xcalc' in an xterm to bring up a calculator.)
-----------------------------------------------------------
Task 3: Create a Header File and Ingest an Image File (5 pts)
From the ENVI Main Menu go to File >> Open Image File.
The "Enter Data Filename" dialogue widget appears. Your newly linked
file 970401_keweenaw.envi should appear in the Files box. If it doesn't,
check the path in the Open Image File widget. The default is
/local/rsi/rsi/envi/data/
Change that to
~/ph330sarlab/*
Click on 970401_keweenaw.envi, then hit Ok (or double click on it).
The "Header Info" widget will appear. Enter the necessary information.
(Samples (columns) = your result. Lines (rows) = your result. Bands = 1.
Data type should correctly default to Byte. Hit OK when done. You have
now created a file called 970401_keweenaw.envi.hdr (i.e., ENVI created
a header file with the same name as the data file, with an appended ".hdr"
suffix).
The "Available Bands List" widget appears.
Good job. You have created a header file and ingested an image.
-----------------------------------------------------------
Task 4: Display an Image (5 pts)
Go to the "Available Bands List" widget. Click on 970401_keweenaw.envi to highlight it, then click on "Load Band" to
display it.
Three windows appear. The large window is the "Image" window. The small windows are the "Scroll" window which shows the location of the subset in the Image window, and the "Zoom" window. The red box in the Scroll window shows the location of the subset displayed in the Image window. The red box in the Image window is the position of the Zoom window. Move it by clicking and dragging.
Cruise around! Check it out!
-----------------------------------------------------------
Task 5: Create a Subset Image of More Manageable Size (5 pts)
Processing goes much faster for smaller images than large.
Go Utilities >> File Utilities >> Create New File >> Standard File . The New File Builder widget appears. Go Import ENVI File and click on 970401_keweenaw.envi . The Create New File Input File widget appears. Click on Spatial Subset. In the resultant Spatial Subset widget, click on Subset by Image. The Subset Function widget appears. A red box circumscribes the image and can by resized by clicking-dragging from the corner, and moved by clicking dragging in the middle. Enter 400 in the Samples and Lines boxes. Click-drag the red box to the far upper right corner of the window. This is the subset you will create. Hit OK, then OK in the Spatial Subset widget, then OK in the Create New File Input File widget. The New File Builder widget appears. Type "subset.envi" in the Enter Output Filename box, then OK.
-----------------------------------------------------------
Task 6: Contrast Stretching - Stretching the Digital Numbers to Highlight Features of Interest (5 pts)
We can adjust the distribution of brightness in the image to highlight features of interest. Histograms of scene DN help you understand the statistics of the your image.
Click right mouse button (b3) inside the Image window. A grey "Functions" box should appear in the upper left corner of the Image window. This box is full of functions (some of which are also available on the Main Menu.)
Go to Functions >> Display Enhancements >> Interactive Stretching
A contrast stretch tool box appears. The left of the two curves is the Input_Histogram of the backscatter intensity values in either the Scroll, Image or Zoom box (go to Histogram_Source to determine which). The histogram reports the number (y-axis) of pixels with a given brightness (digital number, DN; x-axis). The Input_Histogram also indicates the type of stretch used (Linear, Gaussian, etc.). The numbers along the x-axis are the image digital numbers (0-255). The numbers in boxes next to the red and green lines show the values between which the data are stretched (using the default, they are the same).
Stretch_Type and Histogram_Source are perhaps the most immediately useful tools; experiment with them. Go to Histogram Source and select Zoom. Observe how the Input_Histogram changes as you change the Historam_Source between Image, Zoom and Scroll (you are changing the sample population). Most importantly, observe how the histogram changes as you reposition the red zoom selection box over land, ice and water (while Histogram_Source is set to Zoom).
Question 6. Select Histogram_Source >> Scroll. The input histogram will appear as a two-humped distribution of digital numbers. Based on your observations of the distribution of backscatter intensity (distribution of grey values in the image), to which hump would you assign each of the three landscape classes (land, water ice)? Draw a rough sketch of the Input Histogram, and annotate the humps appropriately with land, water and ice.
-----------------------------------------------------------
Task 7: Output a JPEG Image (5 pts)
ENVI can conveniently output image files in various formats. JPEGs are handy because they are space efficient (they use a data compression scheme).
Check your current output directory. Go to
Main Menu, System >> Change Output Directory.
The Change Ouput Directory widget appears. Hit Choose. The Select Directory
widget appears. If the Path box at the top shows the correct directory
(the ph330sarlab subdirectory in your home directory), hit OK. Otherwise,
enter the path to your working directory (~/ph330sarlab).
~/ph330sarlab (for students with accounts on bio/geo machines)
or
/home/geostu/your_home_directory/ph330sarlab
Then hit OK in the Change Output Directory widget.
Display the new subset.envi image.
In the Image window go to Functions >> Output Display >> Image File >> Output File Type >> JPEG. Hit Choose in the Output JPEG File box. The Output Filename widget appears. Replace the text in the Selection box with "subset.jpg". Hit OK. Hit OK in the Display JPEG File Parameters widget.
This writes a file called "subset.jpg" to your working directory (in this case, your ~/ph330sarlab directory on australis01.ice). To check if it came out ok, you can display it using the xv software. Go to the extra xterm you created above. Type
australis01(16)% xv subset.jpg &
In a few moments, the JPEG image will display on your machine. If you want to manipulate it, hit the right mouse button in the image to bring up the "xv controls" widget.
-----------------------------------------------------------
Task 8: Convolution- Median Filter for Smoothing Speckle and Dynamic Overlay for Image Comparison (5 pts)
Speckle is the salt-and-pepper appearance characteristic of SAR imagery. It results from coherent addition of waves from independent scatterers within the radar footprint. If it interferes with the analysis, it may be treated with various neighborhood averaging filters. By linking two displays, we can compare the results of our filtering operation with the original image.
Go Filters >> Convolutions >> Median.
Select subset.envi from the Convolution Input File widget.
Full Scene or Spatial Subset?
>> Full Scene. Hit OK.
The Convolutions Parameters widget offers you options on the Size of the Median filter, and Image Add Back Value. You will find that these parameters affect the output image. Experiment, or use the default values. See the ENVI Online Help for more info on these or any commands.
In the Convolutions Parameters widget, hit Output Result to Memory, then OK. (You can save smaller images to memory, rather than writing them to a permanent file.)
In Available Bands List, hit New (for a new display), select the top file to display (Median(Band 1:subset.envi), then Load Band.
You now have a nice, median filter-smoothed image of the tip of the Keweenaw Peninsula.
Using the Available Bands widget, load subset.envi into Display 1, create a New Display, and then load the median filtered image into Display 2. Compare the median filtered version with the original. Note how the median filter makes the image blurrier, but that speckle is less distracting.
A more powerful way of comparing the images is as follows. Click the
right mouse button in one of the Image windows to bring up the Functions
box in the upper left corner of the Image window. Go
Functions >> Link >> Link Displays.
Go with the defaults and hit OK. Click the left mouse button on either
of the images. A small window will appear under the cursor showing the
same area from the other image. You can click-drag around do display other
portions of the image. To enlarge the linked display area, click-drag the
middle mouse button. You can clearly see what speckle is if you click on
the median filtered image (Display 2).
Output a JPEG of the median filtered image, naming it "median.jpg".
Question 8: Is the spatial resolution the same between the two images? Please comment.
-----------------------------------------------------------
Task 9: Unsupervised Classification- Automated Image Segmentation Using the Isodata Alogorithm (5 pts)
Note: ENVI has changed this algorithm. Now you must have a 2 band image to use this classifier.
Create a pseudo two band image (so you can fool ENVI).
Utilities >> File Utilities >> Create New File >> Metafile.
Import ENVI File >> subset.envi >> OK.
The New File Builder widget indicates that you have subset.envi selected
as "Selected ENVI files for New File."
Repeat the process to ingest that band a second time.
Import ENVI File >> subset.envi >> OK.
Enter Output Filename: subset2band.envi
"Image segmentation" refers to lumping an image into feature classes. A particular landscape feature (e.g., hardwoods, rough ice, calm water) will tend to have a characteristic mean and variance of brightness values (this is essentially why our eye can distinguish among these feature classes, in combination with the difficult to quantify concept of "context"). The Isodata algorithm used here examines the data set, defines n-number of hypothetical feature class means based on image statistics (mean and variance), then assigns each pixel to those classes based on the DN of the pixel.
Go to Classification >> Unsupervised >> Isodata.
Classification Input File widget >> subset2band.envi, hit OK.
Number of Classes: Min (5), Max (10) (go with the defaults).
IsoData Parameters widget, Output Result to Memory, then OK.
Open a New Display (Display 3) (Available Bands List widget, select
the top item on the list (the item with the highest "M"emory integer, and
hit Load Band.)
A color coded image should appear, dividing the scene into 5 to 10 classes. Go to Functions >> Link Displays. Unlink existing links, then create a new Linked Display between the median filtered image (Display 2) and the classified image.
Output a JPEG of the classified image, naming it "classified.jpg".
Question 9: Comment on the correspondence between the median filtered image and the classified image based on your visual comparison of the the two. Does it seem to be a good correspondence? Do any areas/features appear to be potentially misclassified?
-----------------------------------------------------------
Task 10: Color Tables: Exploiting Human Color Perception (5 pts)
Some features are easier to see if we apply color to the grey scale image. Boundaries tend to me more clear, but the meaning of the colors can be more confusing.
Go to your median filtered subset image. Go to Functions >> Color Mapping
>> Envi Color Tables. Click on any of the color tables on the list, then
hit
Options >> Apply
and see what happens! The colored bar shows how the color assignments
are made, ranging from DN=0 on the left to DN=255 on the right. (On some
keyboards, you scan-click through the list using the up and down arrows.)
Experiment with the Stretch Bottom and Stretch Top sliders. Set the color
table to B-W Linear (black and white, linear stretch). Invert the color
(make a negative) by sliding each slider to the opposite end. Does it enhance
the detail in the ice? (You can do the same thing by going Functions >>
Color Mapping >> Control RGB Planes. The RGB Channels widget appears. Go
to Options >> Invert All Channels. Flip it back and forth. Observe how
features become enhanced and subdued this way.
Find a color table that suits you, then export a JPEG image of it, naming
it "color.jpg".
SAR Analysis questions (5 pts each) (please be brief- no grand theses necessary):
A-1. In the original image (970401_keweenaw.envi), why are Houghton and Hancock bright? I.e., why are they brighter than the lake or surrounding forest?
A-2. Why are lakes such as Portage Lake and Twin Lakes relatively dark?
A-3. Why are the NE-SW trending basalt ridges on the north half of the Keweenaw relatively bright? (E.g., near Gratiot Lake, Lac Labelle, Lake Medora)
A-4. The radar look direction is from east to west. That is, the radar was somewhere east of the Keweenaw Peninsula when this image was acquired. With this in mind, can you hypothesize why the western shore of Portage Lake is brighter than the eastern shore?
A-5. Comment on something you find interesting or cool in this
image. Feel free to export a JPEG called "cool.jpg" that shows me your
phenomenon of interest. (FYI- there's an excellent annotation tool under
Image window Functions >> Overlays >> Annotation. Text as well as arrows,
circles, etc.)
Profiles
ENVI allows you to draw profiles across the image. You will find this
invaluable in interactively interrogating your data. I.e., in understanding
the distribution of data values (dn) in your image, among your different
feature classes.
In an image, go to
Functions >> Profiles >> X Profile.
A Horizontal Profile window appears with an x-y plot. The location
of the profile is indicated by the horizontal line in the Image Window.
The crosshairs in the image window correspond to the vertical red line
in the Horizontal Profile Window. The x-axis (Sample) is the position along
the profile from left to right. The y-axis (Data Value) is the digital
number (DN) of the pixel at a given location along the profile.
Hold the left mouse button down and slide it along the graph in the
Horizontal Profile window. The two value reported are the x-pixel position
and the DN, respectively. Note how the crosshairs box moves across the
Image window, and the Zoom window updates itself accordingly. Can you see
how this will be a powerful tool for data interogation?
Overlay profile plots
In the Horizontal Plot Window, go to
Options >> New Window With Plots
An ENVI Plot Window will appear, with your horizontal plot included.
Now, move the profile line to a new row in your Image Window. A new profile
will appear in the Horizontal Profile Window. In that window,
click the right mouse button. The graph should narrow and show the
line number of the profile on the right side.
Depress and hold the left mouse button. This selects the profile for
overlay on the ENVI Plot Window. While holding the left mouse button down,
drag the selected Horizontal Profile to the ENVI Plot Window. The two graphs
should now appear overlayed on the same plot.
You can edit the lines to improve visibility. In the ENVI Plot Window,
go to
Edit >> Data Paramters
The Data Paramters Widget appears. Select one of the lines in the Available
Data Plots field (highlight it). Then change the color by selecting a color
from the list under the Arrow box next to the Color indicator.
Color Arrow >> Items 1:20 >> Green. Then hit Apply.
Now you should have one white and one green graph in the plot, thus
improving your interprebility.
Note that its almost impossible to rlogin in from and display to an xterminal (or 'xterm'). You need to be sitting on a networked Sun computer that has its own cpu. Xterminals are recognized by having a login prompt with an xwindows background of some kind. Stand-alone computers have a 'console' login prompt, without the xwindows-style background. (PACES labs are xterminals, I believe.) (Another note on rlogin and ENVI. On my machine, some of the parameter text in the widget boxes does not display properly; the lower half is truncated. I'm able to work with it like this, but if its too problematic, please go to Dow.)
(Mouse conventions. The default for clicking is the left mouse button.
Some functions also middle and right mouse buttons.
Remote Login Procedure (rlogin):
You will be rlogging into australis01.ice.mtu.edu from your machine (actually, log into any of australis01.ice.mtu.edu through australis14.ice.mtu.edu. Be sure you have 2 digi ts after australis (e.g., australis02.ice.mtu.edu, australis08.ice.mtu.edu, etc) . "**" indicates Unix commands you must enter.
your_machine% xhost australis01.ice.mtu.edu
australis01.ice.mtu.edu being added to access control list
(This says "let others display on my display device".)
your_machine% rlogin australis01.ice.mtu.edu
Password:
(use your ss# the first time, then change your password using the 'yppasswd' command.)
Once your rlogin connection has been made, you will be in an xterm with a "login" prompt. To tell australis01.ice where to display with ENVI, execute the setenv command:
australis01(58)%setenv DISPLAY your_machine:0.0 or,
australis01(59)%setenv DISPLAY=your_machine:0.0
One way to determine what to specify as the DISPLAY variable is to type
on your remote machine
phyfac24% echo $DISPLAY phyfac24:0.0
This shows me that I'm working on phyfac24:0.0 and this would be what
I put after DISPLAY in the setenv command.
If you are unceremoniously ejected or ENVI hangs (unlikely), try killing the process as follows:
australis01(6)% ps | grep idl
5542 pts/0 S 0:00 grep idl
5465 pts/21 S 0:04 /local/mantle.geo/rsi/idl/bin/bin.solaris2/idl -rt=/loc
australis01(7)% kill -9 5465
(ps: list active processes;
| grep idl : find those containing expression "idl";
kill -9 5465: kill IDL/ENVI immediately referenced via process id)