Reprinted from Techniques (February 1999) with permission from the Association for Career and Technical Education (ACTE). For more information about ACTE and Techniques, see http://www.acteonline.org.

ccountability is the hallmark of the 1998 Carl Perkins Vocational-Technical Education Act. Probably no other aspect of the new legislation has more long-term significance for this field of education or presents a greater challenge for successful implementation. In return for granting states greater flexibility in administering and allocating federal funds, Congress has said forcefully that it expects to see results. Career and technical education must demonstrate its contribution to student achievement, program completion, placement in postsecondary education and the workforce and improved gender equity in program offerings. Here’s a look at the new rules. New accountability requirements comprise three features:

1. States must develop systems that monitor performance using four core indicators—student attainment of challenging state-established academic and technical skill proficiencies; student attainment of a high school diploma, equivalent or postsecondary degree or credential; postsecondary education or advanced training placement, retention and completion or placement in the military or employment; technical program participation and completion leading to nontraditional employment.
   
   
2. States must establish and are accountable for the indicators and their performance levels (which must be negotiated with the Education Department). If states fail to reach their performance levels, they will undergo state and local program improvement plans, and federal funding may be withheld.
   
   
3. States will report annual performance to the Education Department, which in turn will report to Congress and the public. The law stipulates that the national report summarize performance results state by state.

Three must-do's

Any problems surrounding the implementation of Perkins accountability requirements may incorrectly paint educators as resistant or unresponsive in the eyes of policy-makers. The only way to avoid that (and a potential loss of program support) is to cover these three bases:

Define workable indicators and performance levels—The core indicators specified by Perkins must be translated into operational measures that clearly tell local providers of career and technical education what information to produce.

With respect to achievement, precisely what are the academic and technical proficiencies students should master? While many states have developed content standards for academic achievement in grades K-12, most do not have similar standards for technical achievement. Additionally, even where academic standards have been established, there are not yet assessment tools for measuring students’ mastery of them.

Furthermore, while other forms of standardized achievement tests are commonplace, in many states there is no testing after grade 10. But the majority of career and technical education occurs in grades 11 and 12. Tenth-grade academic assessment can serve as a baseline, but there is no subsequent testing that would help determine technical education’s contribution to academic achievement.

At the postsecondary level there’s no standardized achievement testing. Furthermore, in most states, neither secondary nor postsecondary systems have valid statewide standardized assessments of technical proficiencies.

In the short run, therefore, many states will have to develop proxies for direct measurement of student achievement. Some possible candidates include grades and certificates of program completion (particularly for programs that use some form of competency-based instruction), but the validity of these proxies will require careful consideration.

States and localities should not have any difficulty producing counts of high school graduates and recipients of postsecondary degrees and certificates. The challenge in making the completion indicator operational is defining a valid underlying population on which to base the completion rate. The problems at the secondary level, like high student turnover, are even more complicated at the postsecondary level where the forms of completion (and the stage in a person’s life when that completion may occur) are much more varied. For purposes of Perkins accountability, postsecondary institutions may want to limit their completion indicators to selected major subpopulations—students 18 to 24, for example—to make definition and measurement more manageable. Such a strategy, however, means not reporting on a substantial part of technical education activity at the postsecondary level.

Placement is a longstanding measure of accountability in career and technical education. But despite years of experience with trying to measure placement in the workforce or the military, technical programs rarely have produced reliable information. Response rates to mail and telephone surveys have typically averaged about 25 percent, too low to produce much confidence in the estimates. States that have had the most success in monitoring job placement of technical education participants are those that have been able to link student records to information contained in unemployment insurance records.

Designing indicators for reporting on gender equity programs may be the easiest task under the new requirements. The law is clear—programs with less than 25 percent of one gender or the other are considered out of balance. Consequently, all that’s required is good data on program enrollment by gender—information that most all secondary and postsecondary institutions have or can readily obtain.

Identify relevant populations—What’s needed is a graduation rate for the subset of students who participate "significantly" in career and technical education. Doing that will be troublesome, to say the least.

At the secondary level, almost all high school students take at least one course in the technical curriculum, so mere participation is not a very useful distinction. And there’s a wide range of technical education participation at the postsecondary level as well. Presumably, the new Perkins calls for documenting the effects of "significant" levels of participation in career and technical education or some coherent cluster of technical and academic courses. What, then, is the degree of concentration that determines whose performance will be monitored? What are the eligible combinations of technical and academic coursetaking? If national guidance is offered, will states be willing to follow it when the national definition is at odds with their own? Right now it’s a "wait and see" situation.

Then there’s the issue of special populations. Are states expected to monitor each indicator for members of this group? The legislation explicitly says states will describe in quantifiable terms the progress of special populations participating in career and technical education and names six groups:

• individuals with disabilities;
• the economically disadvantaged;
• individuals preparing for nontraditional employment;
• single parents, including single pregnant women;
• displaced homemakers; and
• those with limited English skills.

The primary problem is that many states and localities cannot track the individual performance of most special populations. In some instances the problem lies with definition or identification. At the postsecondary level, for example it may be neither legal nor desirable to identity single parents or single pregnant women in student record systems. In other instances, the problem lies in the lack of student record systems that would allow individual monitoring. For example, at the secondary level in two of the nation’s largest states (California and New York), the smallest organizing unit for the state data system is the classroom, a feature that prevents producing any kind of student level data.

In such cases, states will need to create new strategies for tracking the performance of special populations. In some instances, they may be able to piggyback on data collection and reporting already implemented to satisfy requirements for other federal and state initiatives targeting special education, compensatory education or students with limited English proficiency. In other instances, states might employ a sampling strategy to collect performance information on a small but representative number of special populations.

The Education Department’s Office of Vocational and Adult Education (OVAE) is sponsoring a workshop in Kansas City, Mo., Feb. 4-5 to provide technical assistance to state officials who will develop their state’s accountability plan under the new Perkins. The workshop most likely will be the first in a series to help states establish priorities and strategies and prepare final accountability plans to submit to the Education Department.

The National Center for Research in Vocational Education (NCRVE) will join OVAE in providing state-level technical assistance. Workshop updates will be available on the OVAE Web site, http://inet.ed.gov/offices/OVAE.

Develop strategies for local implementation—Perkins requires states to develop their own accountability systems. A critical lesson learned from the 1990 Perkins accountability requirements is that success depends on understanding, acceptance and effective use at the local level. Back then, states devoted considerable effort to developing performance measures and standards, but these measures and standards never became part of local practice in most states. Basic design problems such as lack of assessment tools or adequate management information systems thwarted local adoption. Equally important, most states lacked strategies for adapting state-designed measures and standards to local levels.

Success with the 1998 accountability requirements will depend critically on involving local representatives in the design of state systems and helping local providers of career and technical education learn how to use performance measures to promote program improvement and effective local management. Many educators are accustomed to viewing data as merely something to report to somebody else, not as something that can help them improve curriculum, instruction and student services. (See "Perkins Support Group" for more on NCRVE’s resource on data collection for school improvement.)

The new Perkins does raise other important issues, including developing more effective tools for assessing academic and technical proficiency, integrating technical education’s need for information with the larger secondary and postsecondary information systems of states and localities and establishing credible standards for the quality of information generated by these systems at the local, state and national levels.

For some, the number and complexity of these issues may be sufficient justification for questioning the wisdom of the Perkins accountability requirements. But career and technical education does have a responsibility to demonstrate to students, educators and the public what is accomplished through their substantial investment of time and other resources. Successfully meeting that responsibility depends on understanding clearly the difficult problems that lie ahead and committing ourselves at all levels—local, state and national—to working collaboratively to find solutions.