Viewing Term Statistics

If you have Audit permissions in SpeechMiner ("Audit Events in Calls Grid" is allowed in the permissions of at least one of the Roles applied to your SpeechMiner account), the Topic Manager screen in SMART includes additional information about each term in the Topic that is not visible to other users. The illustration below shows what an open Topic looks like in this case. The table to the right of the list of terms contains the additional information for each term. Only Events that are recognized with at least the strictness specified for the Topic (indicated under Applied Strictness above the list of terms) are included in the data.

Clicking links in this data opens Call Grids listing the calls in each category. You can use the grids to listen to the Topic Events in the calls and audit them for accuracy. This information can help you decide if certain terms should be modified or removed from the Topic, and if additional terms should be added.

Topic Manager screen for users with Auditing permissions

## Overview of the Auditing Data

The auditing data included in the Topic Manager provides information about how often terms in the Topic were found in calls and how accurately SpeechMiner identified them. The table contains the following data for each term:

Column | Description
Percent | The percentage of all the calls in the database in which the term was found
Calls Count | The number of calls in the database in which the term was found
Events Count | The number of times the term was found in the database
Audits | The number of times the term was audited for precision (was rated TP, FP, or SFP)
TP Count | The number of times the term was rated TP during auditing
FP Count | The number of times the term was rated FP during auditing
SFP Count | The number of times the term was rated SFP during auditing
Audits Precision | The percentage of TP ratings among all the audits

## Filtering the Events by Date

You can filter the Event data set by selecting the earliest date to include in the data.

To set the date filter:

1. Above the list of terms, in the Show Calls Since field, select the start date of the data set.
2. Click Refresh Audits. The data is refreshed; only data from calls that took place from the selected date on are included in it.

## Sorting the Terms

You can sort the terms in the list alphabetically.

To sort the terms:

- Click the Text button in the title bar of the list of terms. (Click it a second time to reverse the sort order.)

## Understanding the Term-Auditing Data

Each linguistic Event that is recognized in a call can be audited. This means that a SpeechMiner user listens to the Event and then rates the precision of the identification. The rating can be one of the following:

- TP (True Positive): The Event was correctly identified by SpeechMiner.
- FP (False Positive): The Event was incorrectly identified by SpeechMiner.
- SFP (Sense False Positive): The words were correctly identified by SpeechMiner, but they were used in a sense that does not match the intent of the Topic.

In order to audit an Event, the auditor must play the Event using an Event Grid. After listening to the Event, the auditor rates the accuracy of the Event recognition by selecting the rating in the TP/FP column of the Event Grid. Auditing can be performed in this way either in the SpeechMiner web interface or directly from SMART.

## Opening a Call Grid

You can open a Call Grid directly from the Topic Manager screen. Once the list is open, you can use it to open the Media Player and play back calls. If the grid contains an Event list, you can audit the Event recognition and add or modify the precision ratings of the Events.

Note: For information about working with Call Grids, playing calls in the Media Player, and auditing Events, please refer to the SpeechMiner User Manual.

To open a Call Grid:

- In the table of term information, click the number in the relevant column, as follows:

  Column | List Contents
  Calls Count | A Call Grid containing calls in which the term was found
  Events Count | An Event Grid listing locations in which the term was found
  Audits | An Event Grid listing locations in which the term was audited
  TP Count | An Event Grid listing locations in which the term was audited and rated TP
  FP Count | An Event Grid listing locations in which the term was audited and rated FP
  SFP Count | An Event Grid listing locations in which the term was audited and rated SFP

## Refreshing the Data

While the Topic Manager is open, SpeechMiner continues to process calls. In addition, you or other users may audit Events that were already included in the data. You can refresh the data at any time to incorporate information about newly processed calls and newly audited Events.

To refresh the term data:

- On the right side of the window, click Refresh Audits.

## Managing Terms

Based on the information about the terms that were recognized by SpeechMiner in actual calls, you can modify the words of a term, add new terms, or delete existing terms, as necessary. For example, if a term is not recognized consistently (has a large number of FP ratings), you could add more words to the term to make it easier for SpeechMiner to identify. Similarly, if the auditing data show that the term frequently does not match the intent of the Topic (has a large number of SFP ratings), you could add more words to it in order to ensure it is only identified when it really does match the intent of the Topic. On the other hand, if you think a term with a large number of FPs or SFPs cannot be improved, you might choose to delete it.

For information about how to modify the list of terms defined for a Topic, see Modifying a Topic.

See also

Running the Topic Analysis - Audits Report