Hermetic Word Frequency Counter Advanced 9.41

Hermetic Word Frequency Counter Advanced 9.41 is a program used for scanning a text file, multiple text files, or text on the clipboard, which can also count the number of occurrences of the different words (optionally ignoring common words such as this), and the occurrences of any number of specified phrases. The words (and phrases) which are found can be listed alphabetically or by frequency, with rank and frequency displayed for each one.

The program can be told to allow or disallow words with numerals, hyphens, apostrophes, underscores or colons, to ignore words which are short or which occur infrequently, and to ignore words (e.g., common words) contained in a specified file. It can also be told to count only words in a specified list. This software may be used with text in languages other than English, in particular, with French, German, Italian, Spanish and Portuguese text. It can be used with XML files and with files containing program source code, e.g. C. Results can be written to an output file, optionally comma-delimited for use with spreadsheets.

Major Features:

1. The ability to count phrases as well as words.
2. The ability to scan not just one file but all files in a folder, and optionally in all subfolders, and to return a single report on the frequencies of words and phrases in all files scanned.
3. The ability to specify not only a list of words to be ignored (such as common words in a natural language) but also the ability to specify a list of words and phrases which are to be counted, with all words not in this list being ignored.
4. In the latter case to show, for each word or phrase found, the files in which it occurs.
5. The ability to generate data which can be used to test whether a corpus of text conforms to Zipf's Law.
6. Word
   • The term 'word' usually means a word in a natural language such as English or French, but for this software it has an extended meaning:
   • • In the standard version a word is any sequence of characters consisting of letters from a European language plus (optionally) hyphens, numerals, underscores, colons, periods, apostrophes, @-signs, and forward and backward slashes.
     • In the Advanced Version a word may (optionally) also include ampersands, grave accents, commas and parentheses (the last two thus allowing names of chemical compounds to be treated as words).
   • If the parameter settings allow, a word may begin with an underscore or a colon, and may end with an underscore. A word may not begin or end with a hyphen, apostrophe, period, comma, colon or parenthesis.
   • The Advanced Version allows three additional non-alphanumeric characters within names: commas and opening and closing parentheses. This is so that names of chemical compounds can be treated as words, e.g.: 2,5-dimethoxy-4-(N-propyl-thio) benzaldehyde. If you wish to treat the name of a chemical compound as a word then check the boxes for numerals, hyphens, commas and parentheses.
7. Two Modes of Operation: Count-All and Count-Only
   • The standard version has only one mode of operation: count-all. The Advanced Version has two different modes of operation: count-all and count-only.
   • In count-all mode the software scans one or more files, or text on the clipboard, and counts the number of occurrences of all of the different words.
   • • In count-only mode it scans one or more files, or text on the clipboard, and counts only the occurrences of a specified set of words or phrases.
     • The words (and phrases) found can be listed alphabetically or by frequency, with the rank and frequency of each displayed. In count-only mode the names of the files in which the words and phrases are found can be displayed, together with their frequencies of occurrence in each file.
   • Note that in count-only mode:
   • • The settings in the 'Ignore words' section of the 'Set parameters' window are disabled and have no effect except for the 'Ignore words with fewer than n occurrences' parameter.
     • The presence or absence of a words-to-ignore file (or a list of extra words to ignore) makes no difference, since all such words will be ignored anyway, unless they are included among the words to be counted.
8. Multiple Files
   • Unlike the standard version, which acts only on one file at a time , the Advanced Version can act on multiple files in multiple folders.
   • Files to be scanned can have any filename extension (the part of the filename following the last period), but the files must consist almost entirely of text characters (either 8-bit text or 16-bit Unicode text). For more detail see the paragraph Scannable files in the user manual for the standard version.
   • The software allows you to restrict the files which will be scanned to (a) those having a file extension in a specified list and (b) to files not having a specified extension. This is necessary because there may be, e.g., .js and .css text files mixed up with HTML files and you may wish to exclude them in a scan. (If you restrict the file extensions to one or more then there is no need to specify any to be excluded.)
   • The List files to be scanned operation should be run before doing a scan so that you know exactly what files will be processed.
   • Various types of files are automatically excluded from a scan, in particular, any binary file. This includes Microsoft Word .doc files, whose file formats are not made public by Microsoft. Other files which are automatically excluded are files with the extensions .xls, .pdf and .sys, plus the common graphics and executable files.
9. Words-to-Ignore in non-English Text
   • As stated in the user manual for the standard version, this software may be used with text in languages other than English, including German, French, Italian, Spanish and Portuguese.
   • A words-to-ignore file typically contains common words such as 'in' and 'of'. As in the standard version six files are provided containing common words in English (cwds_en.txt), German (cwds_de.txt), French (cwds_fr.txt), Italian (cwds_it.txt), Spanish (cwds_es.txt) and Portuguese (cwds_pt.txt). These files are in the folder containing the program files (created during program installation), and there is a download link in the Windows Explorer program menu after installation. You can add or remove words as you wish, and words do not have to be in alphabetical order or on separate lines (but the file must consist only of text).
   • If you are scanning multiple files, and those files are in more than one (natural) language, then you can ignore the common words by using a words-to-ignore file which includes the common words in all languages used. For example, there is a file supplied with the name cwds_en_de.txt which contains both common words in English and common words in German.
10. Counting Phrases
    • A phrase is a sequence of words, but if all sequences of words were counted then (in any moderately-sized file) there would be a huge number of them (phrases each of 2, 3, 4 and so on, words), most of which would be of little interest. Thus to count phrases (as distinct from words) you must specify which phrases are to be counted.
    • If you are interested in just a few phrases, they can be added to the 'Extra count-only words/phrases' textbox (they should be separated by commas — see the example below). If there are many phrases to be counted then they should be placed in a text file, and that file specifed by means of the 'Count-only words/phrases file' button.
11. Extra Possibilities for Displaying Results
    • The Advanced Version has four possibilities for displaying the results of a scan which are not available in the standard version. One of these is Zipf data, which produces logarithms of the rank and frequency values. This is not normally needed, but for more details see Zipf's Law.
    • The other three display possibilities (in the list at right, below 'Zipf data') occur only in count-only mode, that is, when a list of words has been specified in the 'Set Parameters' window (either by reference to a file or by means of a short list) and the software is to count only these words, ignoring all other words. Two examples of this will now be given.

• Improved demo version.