He said, she said

Interesting article in Sunday's Fort Worth Star-Telegram about a teacher who was let go for failing too many kids. There's a back-and-forth here that serves as a fascinating case study: a teacher's students are struggling, the principal says that she's failing more kids than other teachers (and besides, in today's environment we don't leave anybody behind), the teacher says she won't raise grades if kids aren't doing the work, the teacher's contract is not renewed.

I call it a case study because it's a perfect example of the pitfalls of subjective grading, which I previously wrote about here. We have no independent data to use in determining who's right and who's wrong. Are the kids skilled at math, and the teacher is grading them unfairly? Are the kids not putting in the effort? Is she a bad teacher? Is she trying to push them harder than other teachers? Is she a hero or a villain? We have no way to know - there's no objective data to work with, no independent evaluation with which to anchor this argument.

Those who responded to my original post have noted the logistical challenges of establishing fair and independent evaluation. And of course they're right - it would be a major undertaking. But if you're this teacher, and assuming you're in the right, wouldn't you want to have that kind of third-party data handy when you meet with the school board? And multiplying this case hundreds of thousands of times over across the country, wouldn't we have a better education system if we separated instruction and evaluation to generate valid information on learning, as opposed to the conflict-of-interest-laden model we have today?

Yes, there are challenges to establishing fair and independent evaluation - but things are only truly impossible when we don't want to do them in the first place.

Happy thought for Friday

A friend sent a link to this video on YouTube; he's familiar with the majority of claims it makes and says that:

With the exception of some of the (possibly extreme) extrapolations near the end, all of these figures are familiar to me and they are fairly well established. Some of them in fact are even rather old... like from Alvin Toffler old even.

One excerpt to kick off your weekend:




Worth viewing the whole thing - the music makes it particularly scary for some reason.

Partnership presentation

I presented our latest research report at NASSMC's annual coalition directors' meeting - wanted to share the PPT here.

Poverty versus school effectiveness

As I mentioned in my previous two posts (here and here), the Education Consumers Foundation has been doing some very cool things with value-added data in Tennessee. In addition to their Value-Added Achievement Awards and school Performance Charts, they also took their composite measures of school performance and plotted them against poverty level. Take a look at the result in the image below (downloadable PDF here):

This scatterplot shows that there is virtually no correlation between poverty rates (as measured by participation rates in free and reduced lunch programs) and the contribution that schools can make to student achievement. What this means is that there are high-poverty schools that are making a major impact on student learning; it also means that there are high-poverty schools doing their children a great disservice. Same goes for schools in affluent areas.

One other interesting piece of information we turned up: when we looked at last year's elementary award winners, we found that eight of the nine winning schools had high percentages of free/reduced lunch participants. Amqui Elementary, the top-scoring elementary school in the state, rang in at 97%.

What this says to me is that we have proof that every student can learn, and just as importantly, every school can teach. The implications of this type of analysis, using a powerful tool like value-added data, are astounding.

Can good data drive school performance?

In my last post, I highlighted one of the primary applications of the Education Consumer Foundation's new School Performance Charts: comparing your elementary or middle school to others in your district. Users can also compare their school or district against others in the state, against the 1998 Growth Standard, or against the current state average. (To try this yourself, go to the elementary or middle chart and give it a spin.)

While this is a tremendous step forward by itself, there's even more here for those with some insider knowledge - people like Mike Cohen, who served with the Knox County Schools as Communications Director at the time when Bill Sanders was piloting his value-added system in that district. Mike pointed out that Knox County, as the pilot site for this innovative model, really bought into the system and continues to actively use that data to drive student achievement - and it's made a real difference in the performance of schools in the county.

Does the School Performance Chart bear that out? Let's compare the Nashville elementary schools (from the last post) to elementary schools in Knox County. Here are the Davidson County/Nashville schools again:

And here's Knox County, pilot site and ongoing proactive user of value-added data:

Comparing the two makes a pretty compelling case: Davidson County's schools range across the spectrum, while Knox County schools are far more concentrated at the upper range of performance. Seems pretty clear that good data, used well, can drive student achievement.

Coming tomorrow: can Title I schools be highly effective?

A game-changing development in education

When I started DeHavilland Associates, it was based on the fundamental assumption that public education needs to engage its stakeholders - the parents, businesses, and other key groups who fund the system and have a vested interest in its outcomes. Lucky for me, then, that I found a client who not only shares that belief, but is empowering these stakeholders in ways that will ultimately revolutionize public education.

I've been working with the Education Consumers Foundation for several months now on their 2007 Value-Added Achievement Awards in Tennessee. They launched the VAAA in 2006 to highlight the use of value-added data, and to highlight the fact that there are schools out there right now that are doing an amazing job of helping students achieve.

Why is this a big deal? Two reasons - one I'll address briefly, the other I'll go into in greater depth.

First reason is the importance of value-added data. In most state assessment systems, school performance ratings don't really measure student performance: what they really do is serve as a proxy rating of their communities. Schools with a mostly affluent population score well; schools in high-poverty areas score poorly.

Value-added systems are different. Thanks to the work of Bill Sanders back in the 80s and 90s, value-added systems are able to strip out external factors like affluence to look strictly and exclusively at the contribution that schools make to student learning. In simple terms, we look at where a student is now (and this does go down to the individual student level), and predict where we expect him to be in one year. If he does better than expected, we can attribute that gain to the school; if he does worse than expected, we can attribute that to the school as well.

So in essence, we have a way to measure school effectiveness that really does look at the impact the school has on student learning, regardless of demographic/socioeconomic factors. Very powerful.

While that's a big deal, it's not news - it's been in place in Tennessee since the 90s. What is newsworthy - and what can change the game in public education - is a new component of the Awards program: the ECF School Performance Charts.

ECF has taken their data from the 2006 Awards, which ranked all elementary and middle schools according to a three year average of reading and math achievement gains, and created a simple, intuitive, and interactive online chart that allows anyone to easily gauge the performance of schools and districts in Tennessee and compare them to others. Take a look at this screen capture, which shows the spread of schools from Davidson County (ie Nashville):



This tool is a game-changer - it provides real information on schools (specifically, what they contribute to a child's education, regardless of external factors such as affluence), and provides it in such a way that any parent, policymaker, or citizen can immediately grasp it.

If your child went to Andrew Jackson Elementary and you saw this chart, how would you react? The school's not at the bottom of the list because it's high-poverty; while it does have an approximate 40% Title I population, the top-performing school in the state (Amqui Elementary, top right of the chart) is 97% Title I. So if that's your kid's school, and every other school in the district is doing a better job of educating its students, how would you react?

And you can do more than just look at the spread of schools across a district: you can compare districts, schools from any location, schools against minimum standards and averages, and schools against the VAAA winners (the highest-achieving schools in the state).

There are a number of other facets worth exploring as a result of this data and these charts, and I'll get to them soon. If you'd like to experience all this for yourself, visit the master project page here, the elementary chart (2006 data) here, and the middle school chart (2006 data) here.

More soon...

Equity or excellence?

Rick Hess and Andy Rotherham bring up an interesting issue in a PDK article titled “NCLB and the Competitiveness Agenda: Happy Collaboration or a Collision Course?” They argue that we can’t have our cake and eat it too: we can either focus on equity or excellence, but we can’t accomplish both simultaneously.

Our last major lurch forward in education policy was NCLB, which is undisputedly a move towards equity, not excellence. We’re disaggregating data so we can monitor the performance of all student groups and requiring that they achieve a minimum standard of competence. (Despite the histrionics of some, the bar is really not terribly high.) There’s nothing in there about excellence –it presents a lowest common denominator approach to education, and offers nothing to those students who have cleared the bar.

The American Competitive Initiative, on the other hand, is supposed to be about excellence. According to the authors, however, the fundamental conflict between these two priorities has resulted in ACI efforts being largely directed outside of K-12 education (much of the money goes to R&D efforts, not to schools or teacher preparation), and have noted that the Initiative has received little support:

While the push for math, science, and engineering has proved popular, the long-term agenda has proved to be a tough sell. The Administration has had trouble winning support for even the modest new expenditures it has proposed. The ACI has fallen prey to political infighting among various members of Congress, and the initial bipartisan support that surrounded the legislation has waned amid quarrels over jurisdictional issues, funding, and specific provisions.

There is a clear conflict between an equity agenda versus an excellence agenda. In terms of setting priorities, funding, and developing accountability systems, it would be very difficult to create a system in which both can occur. Hess and Rotherham note that:

The tension between the equity and competitiveness agendas is made more poignant because influential state-level actors--including key governors, such powerful philanthropies as the Gates Foundation, and such business-oriented groups as Achieve, Inc.--have prioritized high school standards and math and science education. So, while the federal pressure is focusing on bringing up the bottom in K-8 reading and math, these state-level actors are focusing on raising the level of high school achievement. The implication is that policy can do both, but, in practice, the emphasis on gap-closing necessarily shifts attention from higher-end skills, at least in the short term.

What the authors fail to address, however, is that we’re not doing either one very well: in international comparisons, we fall short on both equity and excellence. Andreas Schleicher of OECD notes the following in an article from Education Week’s Quality Counts 2007 report:

Students that did not surpass the most basic performance level on PISA were not a random group. The quarter of American 15-year-olds with the lowest socioeconomic status was almost four times more likely to be among the bottom quarter of performers than the quarter of the most privileged students. It would perhaps be tempting to attribute the performance lag of U.S. students to the challenges that socioeconomic disparities and ongoing immigrant inflows pose to the education system. But among the 41 countries that took part in the latest PISA mathematics assessment, the United States ranks only 10th in the proportion of 15-year-olds with an immigrant background, and all of the countries with larger immigrant shares outperformed the United States.

and:

In fact, international comparisons also highlight important U.S. challenges at the top end of the performance distribution: Only 2 percent of American 15-year-olds performed at the highest PISA level of mathematics, demonstrating high-level thinking and reasoning skills in statistical or probabilistic contexts to create mathematical representations of real-world situations, using insight and reflection to solve problems, and being able to formulate and communicate arguments and explanations. On average across OECD countries, the share of top performers was twice as large, and in Belgium, Japan, and South Korea, even four times as large.

Hess and Rotherham are certainly right in saying that there are conflicts and tension inherent in striving for both equity and excellence in a cohesive and consistent policy. But since most people would undoubtedly agree that both are desirable goals, is there a way to accomplish both? And can’t we at least reach one while we’re trying to figure this out?