Additional Resources for you to Explore
Webinar slides

Step 1.5 - Decide the categories. If you have decided to do analytic scoring, the next obvious step will be to decide on which categories of language behavior you want to assess.  For holistic scales you may want to skip this step. However, even in designing holistic scales, you will need to think about what aspects of the language you will describe in your descriptors for each score level.

Analytic rubrics are by definition based on the idea of providing separate scores for each of multiple categories of linguistic content or language behaviors. The problem for rubric designers is that there are many such possible categories. For example, any of the following could be included:

1. Pronunciation accuracy or level used
2.  Stress timing, rhythm, intonation
3.  Grammar accuracy or level used
4.  Vocabulary accuracy or level used
5.  Collocations
6.  Appropriateness of kinesics, proxemics, facial expressions, or gestures
7.  Use of downgraders
8.  Pragmatics with regard to degree of power difference, social distance, imposition, etc.
9.  Fluency
10.  Organization
11.  Logical development of ideas
12.  Topic coverage
13.  Getting meaning across
14.  Mechanics (capitalization, punctuation, etc.)
15.  Coherence
16.  Cohesion
17.  Register
18.  Style
19.  Successful task completion
20.  Amount of language produced

These twenty categories (each of which could be broken down into separate ratable subcategories) took me less than ten minutes to generate. Imagine what a group of teachers at your institution could come up with.

The central exercise in choosing categories essentially involves being aware of the possibilities and then narrowing down from the many options to three, four, five, or whatever number of categories the teachers/raters will find useful in the particular institution or classroom involved. That is why I say that deciding on language categories will tend to be based on issues of their relative pedagogical and political importance. If a proper needs analysis has been performed recently in your institution or class, the information needed to make such selections may be readily available. But decisions about which categories of language to include in a rubric can also be made at a single meeting if the relevant teachers and administrators are of good will and can meet, brainstorm, and make the decisions.

[from pp. 20-21 of Brown, J. D. (2012). Developing rubrics for language assessment. In J. D. Brown (Ed.), Developing, using, and analyzing rubrics in language assessment with case studies in Asian and Pacific languages (pp. 13-32). Honolulu, HI: National Foreign Languages Resource Center.]