CodeMiner Home Page

CodeMiner
Version 0.99
Released On December 7, 1999 @ 00:59:35
Copyright © 1999 By Amazon Beach Software . All Rights Reserved.

OVERVIEW

This program has two major purposes, to Auto-Document existing C and C++ source code modules and perform Source Code Analysis on these sets of modules which comprise applications and libraries.

Auto-Documentation
Like many fellow software developers, I continuously struggle to maintain complete and up-to-odate source code documentation. As design to release cycles grow shorter, the need for tools which reduce the overall documentation work while ensuring completeness is becoming mandatory. Of course this concept was born several years ago. But it wasn't until the advent of HTML and seeing firsthand the benefits of the first JDK documentation utility JAVADOC that I took this idea seriously. With Auto-Documentation, the following capabilities exist:

Documentation is extracted from actual source code which the more likely to be in synch with the actual code than external post-generation documents.
Developers are encouraged to keep source code comments up-to-date because they translate effortlessly into readable, useable documentation.
Exhaustive hyperlinking of scanned data types, constants and methods transforms source documentation into a inexpensize portable browseable format across the internet, intranets or on individual computers.

Source Code Analysis
Even before I started this program, the ability to perform source code analysis at some level was a key requirement. Thus this application does scan the source code for definitions of types, constants, attributes and methods. While scanning these productions key information is saved and later rendered into the final documentation package for easy viewing. The analysis portion address several key areas including:

Source code metrics such as lines of code, comments, file size, total data types, total functions, total variables, etc.
Source code statistics such as LOC per function, functions per module or class, variables per moduleetc. etc.)
Source code integrity such as missing comments, argument synchronization, duplicate definitions, unknown references etc.
Complexity determination

FEATURES

Standalone Executable
This program is comprised of a single executable. Within the executable are all necessary file handling, parsing and user interface objects thus avoiding the need for Dynamical Link Libraries (DLL's), registry entries etc. Also, due to the executable size (deflated < 400KB, inflated < 1400KB) no installation utility is needed since the executable and supporting GIF files easily fit on a single 1.4MB floppy diskette.

Text Based Archive Files
Creating and managing archive files is accomplished using any text editor thus avoiding the need for a internal bulky GUI based editing dialog/window. Text based archive files are easily readable and allow the user to enter brief descriptions of the packages and modules.

3 Stage Process
This program utilizes a 3 stage process to generate the final documentation package. Each stage is designed to utilize the least amount of memory possible and complete in the shortest time possible.

Stage 1: All modules in all packages of the archive are quickly scanned with all relevant information being temporarily stored in the applications memory. No documentation is generated during this stage.
Stage 2: All module in all specified packages are cross linked internally within the application memory. No documentation is generated during this stage.
Stage 3: All documentation files which comprise the final package are generated including the Archive page, Packages pages, Module pages, Class pages, Index pages and token files.

Definition Rendering
When generating the documentation package (Stage 3), each definition detected is rendered with the module or class it was detected in. Each definition is rendered in a common format to provide consistency and all sub-types of each definition are hyperlinked to their corresponding type definitons (if known). In addition, icons which represent various attributes and/or warning are inserted and include the following:

Visibility (Visible or Hidden)
Module Access (Global or Hidden)
Class Access (Private,Protected,Public)
Storage (Static,Extern,Register)
Modifiers (Constant,Explicit,Mutable,Volatile)
Warnings (Implicit,Missing Comment,Argument Mismatch)
Errors (Orphan Prototype, Method w/o Prototype)

Auto-Commenting
During the initial scan (Stage 1), all comments detected near classes, structures, enumerations, methods, functions and the top of modules are remembered. Then, while writing the document package (Stage 3) these comments are recalled, processed and inserted nearby the definition of the item. When finished, users browsing for selected definitions will see the documentation the developer inserted for public viewing. The comment processor renders all comments and gives special treatment to comment sub-sections that appear to pertain to the following areas:

Method/Function return type sections
Method/Function parameter sections
Author information sections
Revision date/histor sections
Portability information sections

Type Cross-Linking
During cross-linking (Stage 2), all modules in all specified packages are cross-linked at the type level. This additional information is used when writing the documentation package (Stage 3) by inserting hyperlinks to data types from dependent definitions. The following definitions are hyperlinked (if detected):

Super Classes (Parent Classes)
Sub Classes (Derived Classes)
Method/Function return types
Method/Function argument types
Attribute/Variable types
Structure/Union field types
Simple types

Symbol Indexing
When scanning the source code modules (Stage 1), each data type, attribute, variable, method, function etc are detected and bound to their respective class and/or module. Later, when the documentation package is being written (Stage 3), several indexes are generated, sorted and rendered to assist the developer in finding the location, owner or existence of desired symbols. The following indexes are rendered:

Archive Type Index (All Packages)
Archive Function & Method Index (All Packages)
Package Type Index
Package Function & Method Index

Token Listing
At the same time the master indexes are being generated, a separate simple text file containing the names of all type and function tokens in each package is generated. This feature is provided so that users with dynamic editing environments can add developer specific tokens to the editors syntax highlighting scheme. This feature can be quite usefull when modifying code or attempting to reuse code previously written. Each token listing is broken into a section for data types and a section for function/method names.

LIMITATIONS

The primary limitations of this program are related to limitations of all Win16 based applications. This program was developed for Win16 to ensure portability across the entire Microsoft array of Windows operating systems including Win3.1, Win3.11, WinNT3.51, WinNT4.0, Win95, Win98.

Long File Names
Currently this is the most obvious limitation of the program. However it does not imply that source code modules with long file names can not be used. In order to allow scanning of modules with long file names, the short-hand equivalent name must be specified in the Archive File. Of course a descriptive comment can be entered at the same time for clarity sake. Also keep in mind, that the file name is primarily used to load the file and has no other major purpose afterwards.

Wildcard File Specifications
The original concept for the program was for developers, managers or SQA engineers to create the Archive File a single time and then repeatedly scan the source code during the coding, test and release phases to track documentation conformance code size for schedule purposes. Thus the need to specify groups of files by wildcard is not currently supported. The other reason wildcard based file specification is not allowed is to encourage inserting small module descriptions for each entry in the package/archive.

USEAGE SCENARIOS

Although there are many possible uses for this program, (4) major scenarios were envisioned. Hopefully the illustration of each use case (see below) will better help to understand the reason this program was developed.

Software Project Management
This program can be extremely usefull to persons managing software development projects especially in the coding and final release stages. With project size estimates in hand, the project manager can continuously (weekly,monthly,etc) scan all the designated modules in the project and compare the code metrics to estimated size and correlate to effort and schedule. The project manager can also quickly review various metrics and statistics including the size of files, comment:code ratios, and function complexity which might possibly warn of dangerous or unplanned software modules.

(SQA) Software Quality Assurance
To SQA personnel, this program offers the ability to summarize or continuously monitor software projects where metrics, statistics and documentation are concerned. Using the metrics results, correlations from modules and function to development teams can be made and used for future size and effort estimations. At the same time, the results can be quickly scanned to ensure that all source code documentation is being provided per company requirements.

Developer Feedback & Assistance
The individual developer can use this program to track the development of the code they write and monitor how it fits with the code written by others. As the developer completes source modules, they can scan the completed files and look for warnings such as high complexity functions, missing documentation etc. The generated documentation package can also be used by co-developers performing High Impact Inspections to provide a information description of the code they will be inspecting and lead them to modules with results that indicate possible issues (long functions, high complexity, low comment counts etc.)

Reusable & Delivered Software Documentation
Another important use of this program is the support of reuseable libraries and delivering completed software to customer with documentation. In either case, the responsibility to provide full documentation for the developer who will use the delivered code will be essential. Using this program, this phase can completed in a matter of minutes and will be extremely rich provided the original programmers thoroughly documented their code.

SYSTEM REQUIREMENTS

Supported Operating Systems:

Microsoft Windows 3.1x
Microsoft Windows 95
Microsoft Windows 98
Microsoft Windows NT 3.51
Microsoft Windows Nt 4.00

Hard Disk Space:

1Mbyte For Application & Suport GIF Files
Space For Document Package Files (Can Exceed Several MBytes)

System Memory:

Requires Approximately 2 Mbytes for 1 Mbyte of source code

Text Editor:

Notepad
(Or Any Other Editer Capable Of Opening & Saving Raw Text Files)

PROGRAM INSTALLATION

Create Program Directory
Using the directory name of your choice (recommend=CODEMINE), create a directory on your computer where you wish to place the program.

Create Document Directory
Using the directory name of your choice (recommend=CODEMINE\DOCS), create a directory on your computer where the program should write the HTML document files.

Create Shared GIF File Directory
Create a directory below the document directory named SHARED (recommend=CODEMINE\DOCS\SHARED), this is where the shared GIF files will be extracted to.

[Click Here] To Download The Program {CODEMINE.ZIP, 340Kb}
When prompted by your browser, save the file to newly created directory (recommend=CODEMINE). Please wait for the download to complete before going to the next step.

Extract Executable From ZIP Archive
Using WinZip (or any other ZIP based extraction tool), open the CODEMINE.ZIP file and extract the CODEMINE.EXE executable to the program directory created during the first step (recommend=CODEMINE).

Extract Shared GIF Files From ZIP Archive
Using WinZip (or any other ZIP based extraction tool), open the CODEMINE.ZIP file and extract all the GIF files to the shared document directory (recommend=CODEMINE\DOCS\SHARED).

PROGRAM USAGE OVERVIEW

Creating The Package File(s)
The first step required to run the program is to create one (or more) Package Files (PF). This files specify the type of package and location of source code modules which make up the package. Please see the Package File Format section below for details. An example Package File is provided as a starting point.

Creating The Archive File
The next step required to run the program is to create the Archive File (AF). This file controls the entire documentation process. Once this file has been created it will not need to be edited again unless additional packages are required etc. Please see the Archive File Format section below for details. A example Archive File is provided as a starting point. All archive files use the .AF extention.

Running The Program
Once the Package File(s) and Archive File have been created the user can start the program. Simply run the executable CODEMINE.EXE either from the "Start + Run..." menu or via "Windows Explorer". When the program loads an empty list should appear. Select the "File + Open Archive..." menu option and find the Archive File you just created. When the Archive File loads, the list should fill with a node for the Archive, sub-nodes for each Package and sub-sub-nodes for each module. If the package file(s) do not appear, the Archive File must be edited to properly point at the Package File. If the Package File appears but no source code appears then the Package File must be corrected. Once the Archive, Package and Source listing appears correctly the user can run the auto-documentation process.

Scanning The Source Code
To scan all package modules simply press the "Document" button located in the lower-left corner of the application window. As the process begins a Task Dialog will appear tracking the progress of the scanning task. The amount of time required to complete the scanning depends on the total number of package modules specified, speed of the computer and disk write time. For extremely large scans several minutes will be required and several MB of disk space will be used to store the final HTML documenation files.

Reviewing The Auto-Documentation Package
Once the documentation process has completed, simple open your HTML broweser of choice and go to the DOCS directory specified earlier. Open the "index.htm" file. This is the main page for the entire documentation set. From this point, definitons, indexes, class trees, metrics, tokens etc can be viewed.

ARCHIVE FILES

The Archive File (AF) is the main file the user opens when running the application. The primary purpose of the archive file is to specify which Package files (see below) should be load. The archive file also contains all the settings and options which apply to the entire documentation set.

It is recommended that the user first edit/create a Package file for each target in your software build process. When finished each of these Package files will be specified in the archive file.
When the Archive file has been edited as desired the user should open this file with the main program. At this point the user can verify that each Package file was loaded and in-turn all source code modules were found. If an entire package file is not detected, the Archive file must be edited to properly point to the Package file. If the Package File is found and some/all modules are not found the corresponding Package File must be corrected.

Once all Package Files and source code modules are properly loaded, the user can initiate the auto-documentation process by pressing the "Document" button. The time to complete the entire process depends on the number of packages and source code modules being scanned. For a single Package File and a single source code virtually no time is required. As the number of packages and modules increase the execution time also increases.

As of now, the largest documentation set I have scanned contains over 20 Packages which describe nearly 2,000 Header (.H) files and over 1,500 source (.C or .CPP) files, This process on a PIII-500Mhz with 128Mb RAM requires approxiametly 15 minutes and generated nearly 100Mb or HTML. I am continually working to increase the scan rate and reduce the generated HTML disk usage.

The sections below represent the contents of the Archive File. Each entry is listed with a description of the required format etc. Only the areas shown in bold (left side) should be edited.


[OUTPUT]	Required Section
Description=TEXT(Project Archive)	Place a short description here
Output=PATH(C:\CODEMINE\DOCS)	The path where the auto-documentation HTML files should be written
[PACKAGES]	Required Section
Package=FILENAME(C:\CODEMINE\TOOLS.PF)	Enter the 1st Package (.PF) file name here including path
Package=FILENAME(C:\CODEMINE\APP.PF)	Enter the 2nd Package (.PF) file name here including path Repeat these entries as many times as needed to load all desired package files at once.
[IDENTITY]	Required Section
CompanyName=TEXT(Company Name)	Place your companies name here
GroupName=TEXT(Group Name)	Place your group's name here
Webside=URL(Web Address Here)	Place your companies web address here using the full URL such as http://www.aaaaa.com
[CROSS LINKING]	Required Section
LibsToLibs=YES	{YES \| NO} This flag specifies whether definitions from libraries are cross-linked to other libraries.
LibsToApps=YES	{YES \| NO} This flag specifies whether applications are cross-linked back to library definitions
AppsToLibs=NO	{YES \| NO} This flag specifies whether libraries are cross-linked back to applications (rarely used).
AppsToApps=NO	{YES \| NO} This flag specifies whether applications are cross-linked to other applications (only enable when definitions from one application are use in another).
[DOCUMENT COLORS]	Required Section (Use color values 0...255)
Text=RGB(0,0,0)	The color of all text other than the special colors defined below
Background=RGB(204,204,228)	The background color for each HTML page
HyperLink=RGB(0,0,128)	The color of all hyperlinks which have not been visited yet.
VisitedLink=RGB(0,0,128)	The color of all visited hyperlinks.
Heading=RGB(0,32,160)	The text color for extract comment headings
Description=RGB(160,32,0)	The text color for extracted comment section bodies.
TableHeading=RGB(128,128,128)	The background heading color for all tables that are filled.
TableData=RGB(224,224,224)	The background data color for all tables thare are filled.
[VALIDATION TOKENS]	Required Section
Token=ASSERT	Specifies a unique token to associate with runtime/debug error testing
Token=assert	Specifies a unique token to associate with runtime/debug error testing Repeat this entry as many times as needed for all macros and functions that should be associated with error testing.
[ALLOCATION TOKENS]	Required Section
Token=new	Specifies a unique token which represents dynamic memory allocation
Token=alloc	Specifies a unique token which represents dynamic memory allocation Repeat this entry as many times as needed for all macros and functions that should be associated with memory allocation.
[DEALLOCATION TOKENS]	Required Section
Token=delete	Specifies a unique token which represents dynamic memory release
Token=free	Specifies a unique token which represents dynamic memory release Repeat this entry as many times as needed for all macros and functions that should be associated with memory release.
[THREAD START TOKENS]	Required Section
Token=_beginthread	Specifies a unique token which represents a thread entry point
Token=StartThread	Specifies a unique token which represents a thread entry point Repeat this entry as many times as needed for all macros and functions that should be associated with thread initiation.
[THREAD STOP TOKENS]	Required Section
Token=_endthread	Specifies a unique token which represents a thread exit point
Token=StopThread	Specifies a unique token which represents a thread exit point Repeat this entry as many times as needed for all macros and functions that should be associated with thread termination.
[THREAD YIELD TOKENS]	Required Section
Token=Sleep	Specifies a unique token which represents a yield of system resources. Repeat this entry as many times as needed for all macros and functions that should be associated with thread/process yielding.
[SEMAPHORE SIGNAL TOKENS]	Required Section
Token=SIGNAL	Specifies a unique token which represents signaling of a semaphore. Repeat this entry as many times as needed for all macros and functions that should be associated with semaphore signaling.
[SEMAPHORE BLOCKING TOKENS]	Required Section
Token=WAIT	Specifies a unique token which represents a thread waiting for a semaphore to be signaled. Repeat this entry as many times as needed for all macros and functions that should be associated with semaphore waiting/blocking.
[MUTEX LOCKING TOKENS]	Required Section
Token=LockCriticalSection	Specifies a unique token which represents a critical section entry point. Repeat this entry as many times as needed for all macros and functions that should be associated with mutex (critical section) entry points.
[MUTEX UNLOCKING TOKENS]	Required Section
Token=UnlockCriticalSection	Specifies a unique token which represents a critical section exit point. Repeat this entry as many times as needed for all macros and functions that should be associated with mutex (critical section) exit points.
[SIMPLE COMMENT HEADINGS]	Required Section - entries in this section designate source code comment sections as being simple. Simple means the comment is rendered without indentation etc.
AUTHOR=shared/author.gif	This entry designates all source code comment sections "AUTHOR: ..." as simple entities and uses the designated GIF file for a tag leader.
AUTHOR(s)=shared/author.gif	This entry designates all source code comment sections "AUTHOR(s): ..." as simple the same as the entry above. (Please see the included archive file for a more entry ideas).
[COMPLEX COMMENT HEADINGS]	Required Section - entries in this section designate source code comment sections as being complex. Complex means the comment is rendered with the section title on a separate line and bullets are supported etc.
PURPOSE=shared/purpose.gif	This entry designates all source code comment sections "PURPOSE: ..." as complex entities and uses the designated GIF file for a tag leader.
DESCRIPTION=shared/description.gif	This entry designates all source code comment sections "DESCRIPTION: ..." as complex entities and uses the designated GIF file for a tag leader. (Please see the included archive file for a more entry ideas).

PACKAGE FILE FORMAT

The Package File (PF) is the main file used to describe a single build/target in software project. There may be a unlimited number of Package Files specified. For example, let's say your software project consists of (7) libraries and a application. In this example (8) Package Files will be required. When each Package File has been created and edited, the packages themselves will be specified in the Archive File (see above).

It is important to properly edit all portions of the Package File since this file specifies where the source code modules reside, what type of package it is etc. The sections below represent the contents of the Package File. Each entry is listed with a description of the required format etc. Only the areas shown in bold (left side) should be edited.


[SETTINGS]	Required Section
NAME=WIDGETS	Enter a very short code name here which represents the name of the package such as MATH, UTILS, EQUIP, DATABASE etc.
TYPE=LIBRARY	{LIBRARY} - Where the pool of source code represents a static or dynamic library developed within your company or the source code has been provided from another company. {APPLICATION} - Where the pool of source code represents a executable target which can be run by a user. {FIRMWARE} - Where the pool of source code represents a embedded executable. {VENDOR} - When a set of definitions (and source if available) is being scanned to complete all low-level definitions.
DESCRIPTION=INFO(Short Description Goes Here)	Enter a very short description of this package, for example "Math Utilities" etc.
ROOT=PATH(C:\UTILS)	This is a cosmetic entry and path correctness does not matter. This entry displayed on the main Package pages so that users will have general idea of where the package definitions exist.
DEFINTIONS=PATH(C:\UTILS)	This entry specifies the location of the definitions (Header Files) for the package target. This path must be correct and valid. Please note long file names are not yet available so keep this in mind when editing this entry.
IMPLEMENTATIONS=PATH(C:\UTILS)	This entry specifies the location of the implementations (Source Files) for the package target. This path must be correct and valid. Please note long file names are not yet available so keep this in mind when editing this entry.
[MODULES]	Required Section
GROUP=MASKS(.h,.cpp)	This entry specifies the file masks to use for detecting the definition (Header Files) and implementation (Source Files). The file search is conducted in the corresponding directories specified above.

(Revised January 23, 2006 by Webmaster )