TMG and Indexing - Final Thoughts

If you plan to publish a book, you will need to index that book. Therefore, you need to make some decisions, and you should make those decisions sooner, rather than later. Whether you have carefully crafted each tag to fit into the narrative, or whether your narrative is derived from one long memo tag, you need to make some choices.

Your narrative consists of carefully crafted tags without extensive memos

In this case, TMG can automatically create your index from the name variations used in each of those tag sentences. Review our discoveries in our July meeting on indexing. Choose your TMG index options and create name variations that fit your tag sentences and, as much as possible, create good index entries.

Your narrative consists of basic telegraph sentences and narrative tag memo chunks or a single, large narrative memo tag

You must choose a method to index names (and places) in those long memos. As I see it, you have three options.

1. Manually mark each entry for indexing. Pros: (a) You can create the exact index entries you desire; (b) You can create multiple entries for each person or location. Cons: (a) This is tedious; (b) This is prone to error; (c) This lacks flexibility, if you want to change the index format. Warning: If your index combines TMG indexing of telegraph sentences with manual indexing of memos, make sure that the formats match.
2. Use witnesses and roles for all names in your memo field. Pros: (a) Perhaps not error-free, but compared to manual index entries, this may be as close as you can get; (b) All methods require manual work, but this seems to be the easiest; (c) Since TMG is providing the index options, even when changed, the format is consistent for TMG sentences and for memos. Cons: (a) I'm not sure there's a way to create multiple index entries for a single name; (b) It may require imagination to combine the name variation necessary for the index entry with the name variation desired in the narrative paragraph; (c) Your index entries are likely to be inconsistent. Warning: You must create narrative memo tags in manageable chunks - otherwise, you'll be creating dozens of roles to handle dozens of people.
3. Create a concordance file. Pros: (a) This probably yields the most professional result; (b) All indexing options are in the author's hands, since it even indexes the output of TMG's sentence tags. Cons: (a) Once prepared, the concordance file offers a lot of flexibility, but preparing it is laborious; (b) Names in memos must be marked with either the personal ID number or the reference number - less work and less error-prone than manual indexing, but still tedious. Warning: If using the ID number to mark your names, don't renumber your TMG database!

Summary

• Creating a Name-Variation tag for standard index names might be a good idea, regardless of the indexing method you choose.
• Some of the extra work creating the concordance file might be eliminated if you create a given-name-only name variation for each person. (Suggestion, not necessarily a recommendation)
• I'm a telegraph sentence-narrative memo journal report creator, so I would be choosing from among the last three options. This is my personal evaluation.
• If I choose to think about indexing at this point in the game, all three will require tedious memo markup.
• Method (1): Most error-prone; least flexible; good final result.
• Method (2): Least error-prone; reasonably flexible in index formatting; not very flexible in changing name formats consistently; least professional final result,
• Method (3): Most flexible and consistent in name format and index format; most professional final results; requires the most preparation work.

None of these methods is perfect, and all require some work. Personally, I would probably go with Method (3), and am considering beginning the memo markup and the creation of Name-Index variations from this point forward. Regardless of your preferred methodology, if you plan to publish a family history, you must create an index. Choose your indexing method sooner, not later.

The Final Concordance Index

Believe it or not, once the concordance file has been created, the rest of the process is a snap!

• Run your preferred TMG Journal report and export it to Word. If you want places included in your index, select those options, but do not select any of the person index options. (Consider creating a concordance file for places. Just run a List of Places report to Excel and create the two desired columns from the output.) Perform any formatting tasks on your Word document before inserting the index.
• These instructions and illustrations were created with Word 2013. Your Word version may be slightly different. References > Insert Index > AutoMark. Navigate to your concordance file and click Open. Voila! Text values appearing in the concordance file's left-hand column and in your narrative have now been marked with the right-hand column index entry.

Index options in Word showing AutoMark button

• Now, it's necessary to delete all those personal ID numbers. Will my method work for you? Although my citations may include numbers, usually years, in parentheses with no other characters, e.g., (1856), all those entries in my narrative include some modifiers, e.g., (1843-1898) or (b. 1862). Therefore, I can use Word's Find and Replace function to quickly delete the ID numbers in my narrative.
• Select the narrative (Place the cursor anywhere in the document body and Ctrl-A).
• Replace, and with the wildcard option selected, replace all instances of numbers only within parentheses. To eliminate a double space, I also include the preceding space in my "Find what" box.

Word's "Find and Replace" screen

• That goobledy-gook in the "Find what" box tells Word to search for all instances of this text string: space (any number between 0 and 9 appearing any number of times). For example, it will find any of these: (132), (17054), (2648), but will not find these: (b. 1862), (--?--), (1852-1854).
• If you're confident that you use this format only for ID numbers, you can hit "Replace All" and be done. It doesn't take long to go through the document replacing one at a time, though.
• Finally, turn off all formatting symbols (click the ¶ mark) in the ribbon on the Home tab. Place your cursor where you want to insert the index. References > Insert Index > Make all format modifications > OK. It's done - and it's beautiful!

Final index for this short test narrative

Would you like my final thoughts on indexing?

Creating a Concordance File with TMG

Now that I know what I want my concordance file to look like, I want an easy way to create it. The first problem is selecting the names that belong in this file. I do not want to create a concordance file from every name in my database. What I want is a file that includes only those people appearing in my narrative report.

1. I need to select all the people named in my report for addition to the concordance file. If my memo fields name people not in my TMG database, those names must be added to the concordance file manually. Personal opinion: For reporting purposes and for research purposes, all those people, related or not, should be part of the TMG database.
2. I need to include all name variations for each person in the concordance file.
3. I need to include the database ID number for each person.
4. I need to include the years of birth and death, if known, for each person.
5. My final result must be a two-column Word document.
6. The left-hand column must include all the various ways in which a person's name might appear in the narrative report followed by that person's ID number in parentheses.
7. The right-hand column must appear in this format: Surname:Given name variation (yyyy-yyyy).

I know of no TMG report that will create this result in one step. Excel's CONCATENATE function, though, should let me create the desired column contents, and those columns may be copied and pasted into my Word document.

Choosing the Report

The two logical report possibilities are the List of People report and the List of Names report. Column options for the List of People report allow only primary names, and since we want all name variations included in our concordance file, our best report option is the List of Names report. The Focus Group, the Project Explorer, and "All names in the project" options for the subjects of this report all allow one to include all name variations, and the Excel output allows a lot of flexibility in creating the final column contents.

Report Definition screen - List of Names report

Concatenating the first two output columns will yield the desired left-hand column for the concordance file. The List of Names report does not include a year-only option for the birth and death dates, so Excel's handy RIGHT function will create columns with birth and death years only. Concatenating columns 3-5 and the 2 trimmed columns 6-7 will produce the desired right-hand column of the concordance file. (No, this is not a spreadsheet how-to post. If you're unfamiliar with these functions, there are lots of complete explanations already online.)

Report Options - Output columns

Selecting the People

It's easy enough to select all the related individuals in a genealogy narrative. Add the progenitor to an empty focus group, and then add the descendants, the ancestors, the spouses, and finally, all name variations. If your narrative does not mention any unrelated people, your selection stage is done. A good narrative, however, usually names friends and associates. Personal opinion: Those people should appear in your index, too. Here's how I add those people to my focus group.

• My selection method: With your focus group as the starting point, change the Temporary flag (You do have one, don't you?) of all the members to X. Now run a List of Events report identifying all associates of the focus group members. These people will be attached as witnesses to some event in which your focus group member is a principal.

Report filter for List of Events report

• On the Secondary Output tab of the Report Options screen, change the Temporary flag value of all witnesses to match that of those in the focus group.

Secondary Output tab of the Report Options screen

• Now, it's easy to filter the Project Explorer for everyone with this Temporary flag value, select all, and add to the Focus Group. Once you confirm that all name variations for each individual have been added to the Focus Group, the selection process is complete.
• Do you have an easier method? I would love some ideas!

Running the Report

• I run the report using the Focus Group as the subject. The resulting Excel file is CONCATENATEd and RIGHTed to create two columns with the desired content - and the work is still not done. ☹
• Given names only should be marked for indexing, so entries for them need to be created.
• Entries need to be created for names associated with generation numbers; e.g., Samuel2 Ward.
• Index entries need to be standardized.
• Once both columns are satisfactory, copy them and paste them into a Word document. Save the Excel file. If you modify your TMG narrative report, you can add new names to the concordance file with less effort than creating a new one. If you want to change your index entry format, it's easy to do in Excel.
• The font family used in these columns should match that in your TMG narrative report. Once that is finished, so is the concordance file. Whew! What now?

What on Earth is a Concordance File?

One of the newer resources for TMG users is Michael J. Hannah's online book, "Customizing TMG - Using It My Way." It's amazing how many different ways one can solve a problem with TMG! Michael and I have frequently chosen different methods, but I always learn something from his choices. Since Michael was one of the contributors to the TMG-L thread on indexing, I checked out his section, "TOC and Indexes." Surprisingly, that section did not mention roles. Instead, the index method used a concordance file.

What on earth is a concordance file? As Michael's entry states, "The concordance file is a Word document containing nothing but a single, two-column table and no text outside the table. The first column contains a word or a phrase to be searched for throughout the report and indexed, and the second contains the text for the entry." He also gives easy-to-follow instructions on using that concordance file to create an index in a Word document. I followed them, and quickly discovered a problem. I had a lot of different Samuel Wards in my short test narrative - and each mention of the name "Samuel Ward" resulted in an index entry for every one of those men! I needed to rethink my concordance file, the TMG report I used to create it, and ... I also decided to consult the Chicago Manual of Style and Robert Charles Anderson's The Great Migration index to learn a few indexing principles. After all, it's very difficult to create something when you don't know what it is you want to create.

• I needed a TMG narrative report and a concordance file that uniquely identified each individual.
• TMG allows the inclusion of each individual's ID number and/or reference field. My use of the reference field is not consistent, so I opted to include the ID number in my reports.
• The ID number is enclosed in parentheses, so I did the same after every name in my memo field. A concordance file only marks the first appearance of the name in each paragraph, so I didn't worry about marking each individual's name more than once.

Tag entry screen showing ID numbers appended to names in the Memo field

• With the unique identifier problem  solved, I now needed to determine what my index entries should look like.
• Keeping ID numbers in my index would be confusing for readers, but a lot of unidentified Samuel Wards in the index won't be helpful either. I decided I wanted to include the life span; e.g., Samuel Ward (1781-1835).
• What about indexing women's names? Under both maiden and married names? Under maiden names only? CMOS offered some options and TGM include example possibilities.
• What about instances of a.k.a. names or alternate spellings in the text? Should I index those and use a "see" reference; e.g., "Howard, Sucha (see Hayward, Susannah) (1792-1871)? Or should these name variations appear in their own right in the index with no reference to any other name variation? (Note that the inclusion of quotation marks in the index entries causes problems.)
• What about those many occurrences in a narrative in which only the given name appears? How will I work that into the concordance file?

You will probably choose options that differ from mine, but the important thing is to make those choices in advance. This is what I decided I wanted my concordance file to look like.

Concordance file for Samuel Ward narrative

• The colon following the surname creates an index with levels. Level 1 is the surname; given names are indented and follow the surname entry.
• Women are indexed under both maiden and married surnames.
• If indexed under a married surname, the maiden surname, prefaced by "née", is enclosed in parentheses, a CMOS option.
• If indexed under the maiden surname, married surnames appear, each enclosed in parentheses, in chronological order, a slight variation on TGM.
• A name variation is indexed with a "see" reference to the person's birth name index entry or with variations separated by slashes; e.g., Lovina/Lovey.
• The left-hand column includes all the TMG name variations for each person.
• The left-hand column also includes an entry for a person's given name only. A married woman's given name appears multiple times because I wanted an index reference under each married surname and the maiden surname.

Now, how do I create this concordance file? That is the topic of the next post.

More on Indexing

If you plan on using TMG to create the basis of a 500-page book, then it's a good idea to study the various options in TMG's indexing feature. We covered a lot of those options in our July 2015 meeting, recapped in four parts. If you plan to include extensive and beautifully written narratives in that book, you may want even more from TMG's index. Is it possible to index TMG's narrative memos reliably, consistently, and easily? That's the problem that plagues one of our members, and he brought his questions to the RootsWeb TMG-L mailing list. I followed that message string and decided to test what appeared to be the simplest suggestion: using Witnesses in specific Roles.

My Memo field

My Journal reports usually consist of a telegraph paragraph comprising the subject's BMDB tags and memo fields in my custom Comments tag - and source citations, of course. Since I focus on shorter research articles, I seldom worry about indexing, but if I did, how easy would it be? I could manually enter indexing codes in the above memo, but according to replies by Michael Hannah and Terry Reigel, adding Witnesses with Roles to this memo would result in index entries, too. Let's try it!

Tag entry screen showing Memo with Roles

In the example, Samuel's sons, Samuel, Thaddeus, and Josiah, have been added to the tag as witnesses. In the original memo, only the given names of Samuel Ward's sons were included in the paragraph, and the sons were not entered as a simple list. To replicate this memo, I needed to assign a separate role to each son; then, replace the given name with the role assigned to each son, selecting the "Given name" option for that role. The easiest sequence is: Right click at the point in the paragraph where the name should appear > Role > [Select relevant role] > Given name > and select. If I want to replace "Thaddeus" in the original paragraph with his role, the result of this sequence is [RG:Name2], translated as "the given name of the person assigned the role of Name2." The report output is "Thaddeus".

I simply called these roles Name1, Name2, Name3, etc., and it's very easy to add a new Name# if my memo field has several names. One could create specific name variations to be used in indexing; for example, a married woman could have an index name variation such as "Catherine Rollins (née Ward)", which TMG would index as "Rollins, Catherine (née Ward) (1807-1888)". Nice! Of course, since you might not want the name entered in the narrative as Catherine Rollins (née Ward), you would want to use only the first or given name variations in the memo entry.

Tag entry screen showing memo and witness name variation with index output

This looks like a good way to index memo entries. For someone whose narrative memo field may run to ten pages in length, however, there might be a problem. A long memo might require a lot of roles! Score one for breaking up a narrative into multiple small chunks. There is another method to be considered, not as simple, but perhaps even more powerful. Are you ready?