Thursday, January 30, 2014

Selecting and Downloading Raster Data via ftp using Python

My problem was to browse through a large number of Radarsat-2 images on a ftp-server and download the ones falling into my area of interest. Each image is available as a *.zip file and a corresponding *.tif map projected quicklook

There is two main pieces that help solving this problem:

1) You can in fact use gdalinfo to retrieve information about a raster file on ftp without having to download the file itself. The command (which I found documented here ) is, the output is here redirected to the output_tempfile:

gdalinfo /vsicurl/ftp://username:password@ftp.adress.no/outfolder/filename.tif >  output_tempfile


or calling from within Python using os.system with the variables defined before:

os.system(r'gdalinfo /vsicurl/ftp://' + username + ':' + password + "@" + ftpserver + workingfolder + file + ' > ' + output_tempfile)


2) The second step is then to read the centre coordinate from the output_tempfile and if matching download the file. The ftp download is described here , let's see the whole script, which hopefully is self-explanatory with its comments:


import ftplib, os


#DEFINE FTP SERVER PARAMETERS

ftpserver = 'ftp.youradress.no'
username = 'anonymous'
password = 'guest'
workingfolder = '/Out/max/'

#temporary file storing location info from quicklook
output_tempfile = 'C:\Users\max\Desktop\\test.txt'
#destinationfolder for download
destination_folder = 'C:\Users\max\Documents\\'

#Open ftp
ftp = ftplib.FTP(ftpserver, username , password)
ftp.cwd(workingfolder)

#Create a file list of all quicklooks
filelist = []

try:
    filelist = ftp.nlst('*.tif')
except ftplib.error_perm, resp:
    if str(resp) == "550 No files found":
        print "No files in this directory"
    else:
        raise

for file in filelist:
    
    #get quicklook file info and save in temporary file
    os.system(r'gdalinfo /vsicurl/ftp://' + username + ':' + password + "@" + ftpserver + workingfolder + file + ' > ' + output_tempfile)
    
    #Read from temporary file the centre coordinates of the quicklook    
    search = open(output_tempfile)

    for line in search:
        if "Center" in line:
            longitude_quicklook =  float(line[15:25])
        
            latitude_quicklook =  float(line[28:38])
        
        
    search.close()

    print 'Longitude ', longitude_quicklook
    print 'Latitude ', latitude_quicklook       


    #Check if scene is contained in area of interest, divided in two areas since not rectangular
    if ((-23.0 < longitude_quicklook < 67.0) and (71.0 < latitude_quicklook < 90.0)):
        contained = True
        print 'true'
    elif ((-33.0 < longitude_quicklook < 14.0) and (66.0 < latitude_quicklook < 71.0)):
        contained = True
        print 'true'
    else:
        contained = False

 
    zipfile = file[:-7] + '.zip'
    zipsavefile = destination_folder + file[:-7] + '.zip'
    
    print zipfile
    print zipsavefile
    
    #If file contained in area of interest, download the file
    if contained == True:
        print 'transferring ', zipfile
        ftp.cwd(workingfolder)
        ftp.retrbinary('RETR ' + zipfile, open(zipsavefile, 'wb').write)
        print 'transferred ', zipfile
        
    #Close ftp connection    
    ftp.quit()
        
#Remove temporary file
os.remove(r'C:\Users\max\Desktop\test.txt')

#End

No comments:

Post a Comment