HOWTO Drifter Data Processing - Revision history

Kirk at 15:22, 18 July 2009

2009-07-18T15:22:47Z

Kirk at 15:15, 18 July 2009

2009-07-18T15:15:15Z

Kirk at 22:41, 17 July 2009

2009-07-17T22:41:48Z

Kirk at 22:27, 17 July 2009

2009-07-17T22:27:32Z

Kirk at 22:26, 17 July 2009

2009-07-17T22:26:30Z

Kirk at 22:23, 17 July 2009

2009-07-17T22:23:20Z

Kirk at 22:22, 17 July 2009

2009-07-17T22:22:43Z

Kirk at 22:21, 17 July 2009

2009-07-17T22:21:44Z

Kirk at 20:31, 17 July 2009

2009-07-17T20:31:16Z

Kirk: version 1.0

2009-07-17T20:30:53Z

version 1.0

@@ Line 29: / Line 29: @@
 Notes:<br>
 <ul>
-   <li>Use the [http://pacificgyre.com http://pacificgyre.com] website (log: <i>ohlmann</i>, pass: <i>drifterguy</i>) if you need to get an estimation of
+   <li>Use the [http://pacificgyre.com http://pacificgyre.com] website if you need to get an estimation of
   		 when the drifters went in and came out of the water (look at in-water flag values), or go to
               [http://www.icess.ucsb.edu/drifter/realtime/BETA.php http://www.icess.ucsb.edu/drifter/realtime/BETA.php] and enter custom times.

@@ Line 39: / Line 39: @@
     <li>Be sure the <b>IRIDIUM</b> value in it is set correctly!<br>
       If IRIDIUM is set, you may want to check the settings of <b>REMOVE_GPS_01</b> (to set GPS fix quality flags=1 (bad), to have in-water flag value of 0 so they effectively get skipped) and <b>CONVERT_GPS_2_TO_2</b> (to set GPS fix quality flags=2 (2D), to have in-water flag value of 2 to aid in eliminating them in the next section) in the <b>convert_Iridium_reports_to_Microstar_format.m</b> file.
-    <li>Data will be saved in <b>..../kirk/data/Drifter_Anal/2_data/<i>yyyymmdd</b> folder as a <i>yyyymmdd_drifter_dat.log</i> file.
+    <li>Data will be saved in <b>..../kirk/data/Drifter_Anal/2_data/<i>yyyymmdd</i></b> folder as a <i>yyyymmdd_drifter_dat.log</i> file.
     <li>Now, move that folder into a 'deployment type' folder (e.g. .../kirk/data/Drifter_Anal/2_data/<b>SBCLS</b>, and will hereafter called the <b>"working folder"</b>)<br>
 </ul>

@@ Line 253: / Line 253: @@
 <li>Have a beer.
 ----
 <h2>IMPORTANT NOTE ABOUT IMAGEMAGICK</h2>
+In order for the plotting codes to work, you must have [http://www.imagemagick.org/script/index.php ImageMagick] (a free suite of powerful software for image manipulation from the command line) installed.<br>
-</ol><!-- from way near top -->
+Here's instructions on installing on a Mac via MacPorts (since the scripts are currently setup for ImageMagick programs to be in that path):

@@ Line 199: / Line 199: @@
 <h2>Folder: .../kirk/user/3b_plot_drifter_data</h2>
 <b>Purpose: Generate the 4 different types of plots:  All tracks, all tracks animated, individual tracks, and summary plots.</b>
-<div style="border:1px solid red;background-color:#FFEEEE;padding:1px;">IMPORTANT!: See [[#IMPORTANT_NOTE_ABOUT_IMAGEMAGICK ImageMagick]] note below!</div>
+<div style="border:1px solid red;background-color:#FFEEEE;padding:1px;">IMPORTANT!: See [[#IMPORTANT_NOTE_ABOUT_IMAGEMAGICK|ImageMagick]] note below!</div>
 <li>Edit <b>deployment_settings.m</b>:

@@ Line 199: / Line 199: @@
 <h2>Folder: .../kirk/user/3b_plot_drifter_data</h2>
 <b>Purpose: Generate the 4 different types of plots:  All tracks, all tracks animated, individual tracks, and summary plots.</b>
-<div style="border:1px solid red;background-color:#FFEEEE;padding:1px;">IMPORTANT!: See [http://#IMPORTANT_NOTE_ABOUT_IMAGEMAGICK ImageMagick] note below!</div>
+<div style="border:1px solid red;background-color:#FFEEEE;padding:1px;">IMPORTANT!: See [[#IMPORTANT_NOTE_ABOUT_IMAGEMAGICK ImageMagick]] note below!</div>
 <li>Edit <b>deployment_settings.m</b>:

@@ Line 7: / Line 7: @@
 <li>Once proficient, it takes about 20 min to process a typical day's deployment.
 <li>Folders/files referenced here can be found on the ICESS network at '''/home/locl/users/kirk/''' (hereafter referred to as '..../kirk/') in the '''data''' or '''user''' subfolders.
-<li>You should copy the ''kirk'' folder contents to your local computer and run the scripts there instead of on the server.
+<li>You need to copy the <b>kirk</b> folder contents to your local computer and run the scripts there instead of on the server.
 </ul>
 <br>
@@ Line 198: / Line 199: @@
 <h2>Folder: .../kirk/user/3b_plot_drifter_data</h2>
 <b>Purpose: Generate the 4 different types of plots:  All tracks, all tracks animated, individual tracks, and summary plots.</b>
 <li>Edit <b>deployment_settings.m</b>:
@@ Line 252: / Line 254: @@
 </ol><!-- from way near top -->

@@ Line 1: / Line 1: @@
 <h1>Kirk's HOWTO on processing Drifter data</h1>
 <br>
 <h2>Follow these steps to process 'raw' drifter data from [http://www.pacificgyre.com pacificgyre] to final posting on our website:</h2>
 <br>

@@ Line 103: / Line 103: @@
      <li>Here's an example of both in-water flags and redeploy numbers changing for one drifter. This initial shot is how it looked when loaded (after zooming in):<br>
      [[Image:clean_up_starting.gif]]
-     <li>Here's how it looks after correctly changing in-water flags to '0' when it was out of the water, and changing the drifter numbers to '2' for when it was redeployed:<br>
+     <li>Here's how it looks after correctly changing in-water flags to '0' when it was out of the water, and changing the drifter numbers to '2' for when it was redeployed (the track which heads due north):<br>
      [[Image:clean_up_ending.gif]]
    </ul>
@@ Line 109: / Line 109: @@
 </ul>
-<li>Again, when done cleaning up, be sure to press the <b>SAVE</b> button to write your changes out to file.
+<li>Again, when done cleaning up, be sure to press the <b>SAVE</b> button to write your changes out to file!
-<li>Take note of the maximum latitude and longitude limits (you'll want them for the plotting portion below).
+<li>Take note of the maximum latitude and longitude limits (you'll want them for the plotting portion below), then press 'EXIT'.
-<li>Go to the folder containing the file you read-in and:
+<li>Go to your working folder and:
     <ol type="a">
-       <li>Delete the file <i>yyyymmdd_drifter_dat.log</i>  [you made a copy of it already, <i>yyyymmdd_drifter_dat.ORIG.log</i>]
+       <li><b>Delete</b> the file <b>yyyymmdd_drifter_dat.log</b>  [you made a copy of it already as <i>yyyymmdd_drifter_dat.ORIG.log</i>]
-       <li>Rename the file <b>yyyymmdd_drifter_dat.MOD.log</b> file (output of the clean_up_drifter_data.m script), naming it <b>yyyymmdd_drifter_dat.log</b> (so its read in next step). The format of the "MOD" file which this script output is different than the "ORIG" one fetched.
+       <li><b>Rename</b> the file <b>yyyymmdd_drifter_dat.MOD.log</b> file (output of the clean_up_drifter_data.m script), to <b>yyyymmdd_drifter_dat.log</b> (so it's read in next step).
-   </ol>
-<li>The output format is:
+      <li>The format of the "MOD" file which this script output (different than the "ORIG" one fetched) is:
 <pre>
 #1 drifter ID
@@ Line 139: / Line 139: @@
 15723122 2008 10 20 16 19 43 34.406943 -119.691051 13 09 57348 018 -065 0
 </pre>
+   </ol>
 <!-- ---------------------------------------------------------- -->
@@ Line 145: / Line 145: @@
 <b>Purpose: This is the step which takes the data and calculates all the derived products (e.g. dz/dt, u & v velocities, etc.) and outputs a standard formatted file of good data (optionally in interpolated round 10 minute steps).</b>
-<li>Edit top portion of '''process_drifter_data.m''' to set as appropriate:<br>
+<li>Edit top portion of <b>process_drifter_data.m</b> to set these as appropriate:<br>
 <pre>
 %SET **ONE** OF THESE!
@@ Line 164: / Line 164: @@
 </pre>
-<li>Run 'process_drifter_data.m' to process all drifters from a single deployment.<br>
+<li>Run <b>process_drifter_data.m</b> to process all drifters from a single deployment.<br>
 Notes:<br>
 <ul>
-    <li>Will generate '<i>yyyymmdd_drifter#</i>_summary.log' files (or '<i>yyyymmdd_drifter#</i>_TEN_MIN_summary.log' if TEN_MIN_ROUNDING is set) with this format:
+    <li>Will generate '<i>yyyymmdd_drifternum</i>_summary.log' files (or '<i>yyyymmdd_drifternum</i>_TEN_MIN_summary.log' if TEN_MIN_ROUNDING is set) with this format:
 <pre>
           #1 drifter ID
@@ Line 226: / Line 226: @@
     <ul>
        <li>There are now 5 summary plots and one <i>yyyymmdd</i>_all.png plot in your working folder.
-       <li>Use what you see to tweak the values on the deployment_settings.m file (note: if you didn't use bathy in the processing, it doesn't matter what the z_rate or max_z are set to)
+       <li>Use what you see to tweak the values on the <i>deployment_settings.m<i> file (note: if you didn't use bathy in the processing, it doesn't matter what the <i>z_rate</i> or <i>max_z</i> are set to)
        <li>When all looks good, you'll want to eventually set the <b>do_all_anim</b> and <b>do_individ_plots</b> to <b>1</b> to make those plots.
     </ul>
@@ Line 234: / Line 234: @@
 <h2>Folder: .../kirk/user/4_make_web</h2>
-<b>Purpose: Generate "stats.php" pages (e.g. start/stop times, average speeds, average depth, number of drifters, etc.) for each deployment in a project [optnl-ish]</b>
+<b>Purpose: Generate "stats.php" pages (e.g. start/stop times, average speeds, average depth, number of drifters, etc.) for each deployment in a project [optional-ish]</b>
-<li>Run 'make_stats_page.m' and select the project folder (e.g. SBCLS)
+<li>Run <b>make_stats_page.m</b> and select the project folder (e.g. SBCLS).
@@ Line 245: / Line 245: @@
 <li>Copy this deployment's folder (e.g. 20050425) to root folder of deployment type (e.g. SBCLS) on the website, which is found at <b>/home/kirk/public_html/drifter/</b>.
-<li>Copy the '*_summary.log' files to the appropriate deployment type subfolder inside .../kirk/dataDrifter_Anal/2_data/AllData/
+<li>Copy all the <i>*_summary.log</i> files to the appropriate deployment type subfolder inside .../kirk/dataDrifter_Anal/2_data/AllData/
 <li>Right-click on that folder you just copied those files to, and chose 'Compress'.
-<li>Move that generated *.zip folder to /home/kirk/public_html/drifter/data/data.
+<li>Move that generated *.zip folder to inside /home/kirk/public_html/drifter/data/data/.
 <li>On web, update /home/kirk/public_html/drifter/mod_date.txt with today's date.
 <li>Check that the [http://drifterdata.com webpage] looks good.