IntelliMetric®: How it Works
To score essays, IntelliMetric® is “trained” with a set of responses with known scores as determined by experts. These papers are used as a basis for the system to “learn” the rubric and infer the pooled judgments of the human scorers. The IntelliMetric® system internalizes the characteristics of the responses associated with each score point and applies this intelligence to score essays with unknown scores.
IntelliMetric® analyzes more than 400 semantic-, syntactic-, and discourse-level features to form a sense of meaning. IntelliMetric® provides a holistic score as well as scores within five major domains:
- Focus and Meaning
- Organization
- Content and Development
- Language Use and Style
- Mechanics and Conventions
These features fall into two major categories:
- Content – Features of text looking at the content covered, the breadth of content, and the support for concepts advanced (e.g., vocabulary, concepts, support, elaboration, word choice). Features pointing toward cohesiveness and consistency in purpose and main idea (e.g., unity, single point of view, cohesiveness). Features targeted at the logic of discourse including transitional fluidity and relationships among parts of the response (e.g., introduction and conclusion, coordination and subordination, logical structure, logical transitions, sequence of ideas).
- Structure – Features examining conformance to the conventions of edited American English (e.g., grammar, spelling, capitalization, sentence completeness, punctuation). Features targeted at sentence complexity and variety (e.g., syntactic variety, sentence complexity, usage, readability, subject-verb agreement).
Based on these and other features, IntelliMetric® identifies the underlying semantic structure for a given piece of writing. Fundamentally, IntelliMetric® synthesizes broader meanings from many more molecular traits. More than 400 features of the text and multiple mathematical models are applied to derive the critical semantic structure.
