Kirk's HOWTO on processing Drifter data

WARNING: THIS PAGE IS NOT DONE

Follow these steps to process 'raw' drifter data from pacificgyre to final posting on our website:

Notes:

Once proficient, it takes about 20 min to process a typical day's deployment.
Folders/files referenced here can be found on the ICESS network at /home/locl/users/kirk/ (hereafter referred to as '..../kirk/') in the data or user subfolders.
You should copy the kirk folder contents to your local computer and run the scripts there instead of on the server.

Folder: .../kirk/user/0_Info

On deployment day: be sure to fill out .../kirk/user/0_Info/drifter_hist.xls (drifter history) with the performance of the drifters to track if there are any consistent problems (Note that this file is now out of date).

Folder: .../kirk/user/1_get_data_from_web
Edit data_dates.m with the start/stop date/times to get.
Notes:
- Use the http://pacificgyre.com website (log: ohlmann, pass: drifterguy) if you need to get an estimation of when the drifters went in and came out of the water (look at in-water flag values), or go to http://www.icess.ucsb.edu/drifter/realtime/BETA.php and enter custom times.
- Keep adding date/times to the bottom of this file (or in the appropriate "deployment type" area (e.g. SBCLS), and commented out), then copy these uncommented deployment times to the top of it to fetch the data. Note, when in the "deployment type" groups, I number the 'dates' sequentially, but when copying one or two of them to the top to fetch, you need to make sure those 'dates' numbering start at 1.
- You don't have to be exact on the times (cleanup happens in following steps), just be sure you have a time at least before or at start and a time after or at ending deployment.
Run get_data.m
- Be sure the IRIDIUM value in it is set correctly
- Data will be saved in '..../kirk/data/Drifter_Anal/2_data/yyyymmdd' folder as a yyyymmdd_drifter_dat.log file.
- Now, move that folder into a 'deployment type' folder (e.g. .../kirk/data/Drifter_Anal/2_data/SBCLS)
Notes:
- If IRIDIUM is set, you may want to check the settings of REMOVE_GPS_01 and CONVERT_GPS_2_TO_2 in the convert_Iridium_reports_to_Microstar_format.m file.
- Data format (column numbers):
```
      #1 drifter ID
      #2 year
      #3 month
      #4 day
      #5 hour
      #6 minute
      #7 second
      #8 lat
      #9 long
     #10 satelites availiable
     #11 satelites used
     #12 com flag
     #13 Secs to fix
     #14 Mobitex RSSI
     #15 in-water flag
```
Folder: .../kirk/user/2_clean_up
Edit clean_up_drifter_data.m with the correct area and deploy_date at the top.

LEFT OFF HERE!

folder: 3_process_drifter_data

<input type="checkbox">Edit top portion of 'process_drifter_data.m'.
Mostly just need to set one of these:

<span class="mygeneral1-identifier">SBCIS_SBCSS </span><span class="mygeneral1-symbol">=</span><span class="mygeneral1-space"> </span><span class="mygeneral1-number">0</span><span class="mygeneral1-symbol">;</span><span class="mygeneral1-comment">%UCSB inner shelf or small-scale CODAR area work</span>
<span class="mygeneral1-identifier">SBC_FULL </span><span class="mygeneral1-symbol">=</span><span class="mygeneral1-space"> </span><span class="mygeneral1-number">0</span><span class="mygeneral1-symbol">;</span><span class="mygeneral1-space"> </span><span class="mygeneral1-comment">%all of Santa Barbara Channel for Large Scale Deployments or SCR</span>
<span class="mygeneral1-identifier">UCSD </span><span class="mygeneral1-symbol">=</span><span class="mygeneral1-space"> </span><span class="mygeneral1-number">1</span><span class="mygeneral1-symbol">;</span><span class="mygeneral1-space"> </span><span class="mygeneral1-comment">%San Diego Imperial Beach</span>
<span class="mygeneral1-identifier">HBDP </span><span class="mygeneral1-symbol">=</span><span class="mygeneral1-space"> </span><span class="mygeneral1-number">0</span><span class="mygeneral1-symbol">;</span><span class="mygeneral1-space"> </span><span class="mygeneral1-comment">%Huntington Beach or Dana Point (DPIO) area</span>
<span class="mygeneral1-identifier">SAR </span><span class="mygeneral1-symbol">=</span><span class="mygeneral1-space"> </span><span class="mygeneral1-number">0</span><span class="mygeneral1-symbol">;</span><span class="mygeneral1-space"> </span><span class="mygeneral1-comment">%Santa Ana River</span>

<span class="mygeneral1-identifier">GEODAS </span><span class="mygeneral1-symbol">=</span><span class="mygeneral1-space"> </span><span class="mygeneral1-number">1</span><span class="mygeneral1-symbol">;</span><span class="mygeneral1-space"> </span><span class="mygeneral1-comment">%1=use bathy from GEODAS CD (manditory if UCSD or SAR is set above), 0=use original brian chinn sbc_bathy</span>

<input type="checkbox">Run 'process_drifter_data.m' to process all drifters from a single deployment
Notes:

'process_all.m' will go thru and run 'process_drifter_data.m' on all subfolders.
Will generate 'yyyymmdd_drifter#_summary.csv' files with this format:

         #1 drifter ID
         #2 year
         #3 month
         #4 day
         #5 hour
         #6 minute
         #7 second
         #8 lat
         #9 lon
    *   #10 u vel (cm/s)
    *   #11 v vel (cm/s)
    *   #12 along velocity (cm/s)
    *   #13 across (offshore) velocity (cm/s)
    *   #14 depth (m)
    *   #15 dz/dt (cm/s)
    *   #16 serial time
        #17 satellites available
        #18 satellites used
        #19 com flag/Receiving status
        #20 Secs to fix
        #21 Mobitex RSSI

    * denotes a derived product

folder: 3b_plot_drifter_data

<input type="checkbox">Edit 'deployment_settings.m' to set max depth, dz/dt, lat/lon and velocity limits on the associated plots.

<input type="checkbox">Edit top portion of 'do_plots.m'.
Mostly, to set any of these:

<span class="mygeneral1-identifier">do_all </span><span class="mygeneral1-symbol">=</span><span class="mygeneral1-space"> </span><span class="mygeneral1-number">0</span><span class="mygeneral1-symbol">;</span><span class="mygeneral1-space">   </span><span class="mygeneral1-comment">%0 = do plot, 1 = skip</span>
<span class="mygeneral1-identifier">do_all_anim </span><span class="mygeneral1-symbol">=</span><span class="mygeneral1-space"> </span><span class="mygeneral1-number">1</span><span class="mygeneral1-symbol">;</span>
<span class="mygeneral1-space">  SHOW_CODAR_TRACKS </span><span class="mygeneral1-symbol">=</span><span class="mygeneral1-space"> </span><span class="mygeneral1-number">0</span><span class="mygeneral1-symbol">;</span><span class="mygeneral1-space"> </span><span class="mygeneral1-comment">%show interpolated 'simulated drifter' tracks based on CODAR data (uses '*_compare.mat' files) for anim plot</span>
<span class="mygeneral1-identifier">do_individ_plots </span><span class="mygeneral1-symbol">=</span><span class="mygeneral1-space"> </span><span class="mygeneral1-number">0</span><span class="mygeneral1-symbol">;</span>
<span class="mygeneral1-identifier">do_summaries </span><span class="mygeneral1-symbol">=</span><span class="mygeneral1-space"> </span><span class="mygeneral1-number">0</span><span class="mygeneral1-symbol">;</span>
<span class="mygeneral1-identifier">SHOW_CODAR </span><span class="mygeneral1-symbol">=</span><span class="mygeneral1-space"> </span><span class="mygeneral1-number">0</span><span class="mygeneral1-symbol">;</span><span class="mygeneral1-space"> </span><span class="mygeneral1-comment">%draw the hourly codar vectors on the anim plot</span>

folder: 4_make_web

Run 'make_stats_page.m' (generates all 'stats.php' pages for a deployment area)
Copy this deployment's folder (e.g. 20050425) to root folder of deployment type (e.g. SDSS) on the website, but only the *.png, stats.php & *.gif files (and 'anim' subfolder).
Copy the '*_summary.log' files to the appropriate deployment type subfolder at E:\User\Carter\data\Drifter_Anal\2_data\AllData
Right-click on that folder you just copied those files to, and chose 'compress to *.zip'.
Move that generated *.zip folder to U:\public_html\drifter\data\data.
On web, update 'mod_date.txt' with correct (today's) modified date.
Check <a href="http://drifterdata.com">website</a> looks good.
Syncronize E:\User\Carter with /home/data318/kirk
Have a beer.

HOWTO Drifter Data Processing

Contents

Kirk's HOWTO on processing Drifter data

Follow these steps to process 'raw' drifter data from pacificgyre to final posting on our website:

Folder: .../kirk/user/0_Info

Folder: .../kirk/user/1_get_data_from_web

Folder: .../kirk/user/2_clean_up

folder: 3_process_drifter_data

folder: 3b_plot_drifter_data

folder: 4_make_web

Navigation menu

Page actions

Page actions

Personal tools

Navigation

Search

Tools