NAVIGATION

Creating
Metadata

Parsing
Metadata

Distributing
Metadata

                       

Introduction

Software

Helpful
links

                       

HOME

 

Parsing your metadata         
Excerpts by Peter N. Schweitzer [pschweitzer@usgs.gov]

Updated on 19990827


In Lesson 1, you learned how to download, install, and use mp from the command prompt.  Lesson 2 will now introduce parsing metadata.

Parsing your record involves more than just typing a command.  As indicated earlier (parsing metadata), parsing with mp is:

  • Error Checking:
    Fortunately, mp checks metadata so you don't have to, saving you hours of review.  The errors are identified as discrepancies between the values/syntax of the record and the metadata content standard (CSDGM).
    Your job is to correct those errors.

  • Output file generation:
    Using mp will allow the user to generate output that can be utilized for data distribution.  MP gives you the opportunity to generate two web based file types, SGML, Text, and DIF output types.   Lesson 3 will examine these files and how  best you can use them..

Learning how to develop strategies that will allow you to efficiently parse metadata is the primary goal for this lesson. 

Following the steps below, you will be introduced to a sample metadata record developed with specific syntax and positional errors.  These errors represent some of the most common mistakes encountered by parsing. 

  • BEFORE YOU BEGIN:  
    Before you begin this lesson, you will need the following:

    Applications:

    • MP  

    • Corpsmet95 (as used in earlier examples)

    • Windows Explorer: C:\Temp directory 

    • Microsoft Notepad 

    • A *.zip compatible decompression software product (WinZip)

    Data Package:

 

[Go To Step 1:  Parse your record]

 


 

  • Step 1:   Parse the metadata record
    There are a number of procedures that ensure successful parsing of metadata. Organization is the best strategy you can employ.  For beginners, it's a good idea to focus on these 4 process steps:

    1. Parse the metadata record

    2. Review the errors

    3. Correct the errors

    4. Do it again...

    Today's lesson may not be provide the most direct strategy for parsing metadata, but it will provide the novice user with the knowledge and abilities needed to complete the task.  Naturally, you are encouraged to develop strategies that best work for you.

     

    • Parsing the metadata record:

      • Ensure "jltreasure.met" is located on c:\temp. If it is not present, follow the instructions presented above: Before You Begin.

      • Open the Command Prompt:



      • Move to the Temp directory by typing at the prompt:

      cd temp

      • Execute mp so that it parses the record "jltreasure.met" and generates an error file titled "error.txt" by typing:

      mp -e error.txt jltreasure.met


(Command Prompt window used with mp)

This command generates the "error.txt" file in the c:\temp directory that contains all the errors in the file "jltreasure.met" detected by mp.

 

[RETURN TO TOP] [NEXT STEP]

 

 



  • Step 2:  Review the error file
    Once generated, you can review any and all errors identified in the metadata record.

    • Open the "error.txt" file:

      • Open c:\temp directory and locate error.txt.

      • Open the error.txt file (double-click the icon).  The file should open in Notepad.

      • Review the errors:


      (Error.txt file listing the errors present after the first parse)

      Notice how many errors there are (as indicated at the bottom of the list):

149 errors: 1 misplaced, 12 too_many, 135 missing, 1 bad_value

While the number of errors seems alarming, it's quite normal, especially for beginners. Don't be discouraged.  Correcting these errors are usually  as simple as changing one value, as you will see...

[RETURN TO TOP] [NEXT STEP]




  • Step 3:  Correcting the errors
    There are a number of different ways to correct errors.  Some individuals prefer to edit their metadata record using Microsoft Notepad.  Others use the application that initially developed the metadata.  Finally, some use a combination of both Notepad and their development app.

    Step 3 will use Corpsmet95 to correct any obvious methods.   Later, in Step 5, you will see how to correct errors using Notepad.

    Correcting errors using Corpsmet95

    • Open Corpsmet95
      1. Open the "jltreasure.met" file located in c:\temp:


    (Corpsmet95 Open dialog box)

    1. Identify any obvious error as indicated by the icon and correct them.  Take a close look at the errors:


    (Corpsmet95 metadata project box)

    1. Familiarize yourself with the errors.  Identify how and why they caused problems.  Some are more obvious than others.
    • Making the corrections:
      Reviewing the metadata record, you will need to make the following corrections:
      • Identification > Description >  Abstract:
        The digitized features identifying the potential locations for the legendary treasure(s) of the pirate Jean Lafitte.
      • Identification > Access Constraints:
        Select "No Access Constraints"
      • Identification > Point of Contact:
        Select "Organization"
                                                                             
      • Data Quality > Completeness Report:
        Information regarding the Jean Lafitte treasure are purely fictional and not based on substantive information.
                                                                             
      • Spatial Reference > Horizontal Coordinate System Definition > Planar 1 > Grid Coordinate System:
        Select the tab titled "Universal Transverse Mercator" and enter "0" for "False Northing"
                                                                            
      • Data Distribution > Distribution Information 1 > Distributor:
        Select  "Organization"

    • Save the corrections:
      Select "Project" from the Corpsmet95 menu and choose "Build metadata file." Make sure you are in the c:\temp directory.  This will over-write the original  "jltreasure.met" file you downloaded.


    (Corpsmet95 Create Metadata File dialog box)

    With the corrections made, it's time to parse the metadata record again...

    While these errors were obvious, it does demonstrate an important features of Corpsmet95:

    • Use the Corpsmet95 symbology (icons like ) to correct errors before you parse.  These icons inform you when a field requires a value

[RETURN TO TOP] [NEXT STEP]

 


 

  • Step 4:  Parse again...
    Once you have corrected the obvious mistakes, it's time to parse the record again.  It's a good idea to make a few corrections at a time.

    By parsing every so often, you learn if the corrections you submitted are valid and you can identify how many errors are left.  

    Remember, sometimes it takes only one or two errors to cause many

    • Just as was it was demonstrated in Step 1, from the command prompt execute mp so that it parses the record, "jltreasure.met" and generates an error file titled "error.txt" by typing:

      mp -e error.txt jltreasure.met


    (Command Prompt window used with mp)

    • MP generates the "error.txt" file again, over-writing the earlier one.  Just as before, you need to open the "error.txt" file (located in the c:\temp directory) and examine the results:


      (Error.txt file listing the errors after the second parse)

      ** GREAT NEWS! **

      As you can see, the error listing went from 149 to 1 error and 1 bad value.  This is fantastic.  Now it's time to correct the last few errors.

[RETURN TO TOP] [NEXT STEP]

 


 

  • Step 5:  Correcting the remaining mistakes
    Just as in Step 3, you will use Corpsmet95 to correct the last mistakes. As you found out, the majority of the registered errors were a result of a number of missing and/or misplaced values.

    These next few mistakes are different.  They are not as easily identified and may require additional tools to correct.  

    Here's what we will do:  

    • Correcting  Corpsmet95 again:

      • From the Corpsmet95 menu, click "File" and  "Open...".  When the open dialog box becomes visible, open the file, "jftreasure.met" from the c:\temp directory.


      (Corpsmet95 Open dialog box)

      • Error (line 163): improper value for Cloud Cover

        • Data Quality > Cloud Cover
          Because Cloud Cover is not a required field, you can delete the current value, "N/A" and leave the field blank... or enter "0" as the value.  

        • You will notice that this error is "hidden," meaning Corpsmet95 did not find this error.  That is because Corpsmet95 does not check of logical consistency (validity of an entry).

    This concludes correcting errors.  While you can parse the file again and receive no errors, you will continue to have the "Upgrade" mistakes.  

    If you wish, you may move on to Step 6

    OPTIONAL CORRECTIONS:
    These next corrections represent errors that do not require corrections to be considered content compliant.  However, these mistakes are recommended.  

    Additionally, these errors also introduce an another method for error correction.

     

    • Review the "Upgrade" errors:
      When mp indicates an "Upgrade," it usually means that something has changed in the CSDGM for which Corpsmet95 has not compensated.  Correcting "Upgrades" are optional, but highly recommended.

      Starting in reverse order, here is how you solve these problems using Corpsmet95


      • Upgrade (line 305): value of Metadata_Standard_Version set to FGDC-STD-001-1998

        • Metadata Reference > Standard:
          Enter "FGDC-STD-001-1998" where in the "Version" field.

     

    • Save your changes:
      Save your changes just as described in Step 3.


    (Corpsmet95 Create Metadata File dialog box)

    • Errors not correctable using Corpsmet95:
      There are two errors that cannot be corrected by Corpsemet95:

      • Upgrade (line 198 & 210): Detailed_Description added for this Entity_Type

        These errors exist because Corpsmet95 failed to add the text "Detailed_Description" above the entries titled "Entity_Type."  Solving this is a little tricky, and will require opening "jltreasure1.met" using Microsoft Notepad.

        Here's how you do it:

        1. Click on the Start Button

        2. Select "Run" from the Start Menu

        3. Type this:
               notepad c:\temp\jltreasure.met

        4. A Notepad window will open with the Jean Lafitte metadata record already loaded.

      • Now that you're in Notepad, we have to search for the entry "Entity_and_Attribute_Information" using the "Search," and "Find" function of Notepad:


      (Notepad Find dialog box)

       

      • Notepad will move to the selected text.  From there, you will add the text:

        Detailed_Description:

        above each heading that reads:

        Entity Type:

        so that your the final product looks something like this (where your edits are highlighted blue):

    Entity_and_Attribute_Information:
           Detailed_Description:
                Entity Type:
                      Entity_Type_Label: Bayou Barataria...
                      ...
        
    Detailed_Description:
                Entity_Type:
                       Entity_Type_Label: Parishes...
                       ...
        
    Detailed_Description:
                Entity_Type:
                        Entity_Type_Label: Raster Grid... 
                        ..
    .

Save the changes you've made because you are going to parse this record one last time...


[RETURN TO TOP] [NEXT STEP]

 


 

 

  • Step 6:  Parsing one last time...
    It's finally coming to an end.  The record has been parsed, checked, and corrected twice.  We shall parse it again, just as done it Step 1 and Step 4


(Command Prompt window used with mp)

Now that the file "jltreasure.met" has been parsed again, we must review the generated error.txt file for errors.

This time, as you can see, there are NO ERRORS!!!!

Believe it or not, but you have successful metadata.  If you would like to see the corrected metadata record, you can download and save the file:

Corrected_jltreasure.met

All that is remains to be done is generate the different file types that best suit your needs.  But don't worry about that!  Why?  Because mp does it all for you, quick and easy!

But that's not until the next Lesson....

LESSON 3: Generating Distribution Files

L A G I C
Louisiana Geographic Information Center