Thursday, January 30, 2014

Selecting and Downloading Raster Data via ftp using Python

My problem was to browse through a large number of Radarsat-2 images on a ftp-server and download the ones falling into my area of interest. Each image is available as a *.zip file and a corresponding *.tif map projected quicklook

There is two main pieces that help solving this problem:

1) You can in fact use gdalinfo to retrieve information about a raster file on ftp without having to download the file itself. The command (which I found documented here ) is, the output is here redirected to the output_tempfile:

gdalinfo /vsicurl/ >  output_tempfile

or calling from within Python using os.system with the variables defined before:

os.system(r'gdalinfo /vsicurl/ftp://' + username + ':' + password + "@" + ftpserver + workingfolder + file + ' > ' + output_tempfile)

2) The second step is then to read the centre coordinate from the output_tempfile and if matching download the file. The ftp download is described here , let's see the whole script, which hopefully is self-explanatory with its comments:

import ftplib, os


ftpserver = ''
username = 'anonymous'
password = 'guest'
workingfolder = '/Out/max/'

#temporary file storing location info from quicklook
output_tempfile = 'C:\Users\max\Desktop\\test.txt'
#destinationfolder for download
destination_folder = 'C:\Users\max\Documents\\'

#Open ftp
ftp = ftplib.FTP(ftpserver, username , password)

#Create a file list of all quicklooks
filelist = []

    filelist = ftp.nlst('*.tif')
except ftplib.error_perm, resp:
    if str(resp) == "550 No files found":
        print "No files in this directory"

for file in filelist:
    #get quicklook file info and save in temporary file
    os.system(r'gdalinfo /vsicurl/ftp://' + username + ':' + password + "@" + ftpserver + workingfolder + file + ' > ' + output_tempfile)
    #Read from temporary file the centre coordinates of the quicklook    
    search = open(output_tempfile)

    for line in search:
        if "Center" in line:
            longitude_quicklook =  float(line[15:25])
            latitude_quicklook =  float(line[28:38])

    print 'Longitude ', longitude_quicklook
    print 'Latitude ', latitude_quicklook       

    #Check if scene is contained in area of interest, divided in two areas since not rectangular
    if ((-23.0 < longitude_quicklook < 67.0) and (71.0 < latitude_quicklook < 90.0)):
        contained = True
        print 'true'
    elif ((-33.0 < longitude_quicklook < 14.0) and (66.0 < latitude_quicklook < 71.0)):
        contained = True
        print 'true'
        contained = False

    zipfile = file[:-7] + '.zip'
    zipsavefile = destination_folder + file[:-7] + '.zip'
    print zipfile
    print zipsavefile
    #If file contained in area of interest, download the file
    if contained == True:
        print 'transferring ', zipfile
        ftp.retrbinary('RETR ' + zipfile, open(zipsavefile, 'wb').write)
        print 'transferred ', zipfile
    #Close ftp connection    
#Remove temporary file


No comments:

Post a Comment