"Reading and math scores have long been the currency of American schooling, and never more so than in the past two decades since the No Child Left Behind Act. Today, advocates will describe a teacher as “effective” when what they really mean is that the teacher’s students had big increases in reading and math scores. Politicians say a school is “good” when they mean that its reading and math scores are high.

So, how much do test scores really tell us, anyway? It turns out: A lot less than we’d like.

For all the attention to testing, there’s been a remarkable lack of curiosity about how much tests tell us. Last spring, for instance, researcher Collin Hitt, of the Southern Illinois University School of Medicine, and two coauthors examined the research on school choice and found a striking disconnect between test score gains and longer-term outcomes. They reported, “Programs that produced no measurable positive impacts on achievement have frequently produced positive impacts on attainment” even as “programs that produced substantial test score gains” have shown no impact on high school graduation or college attendance. More generally, they observe:

The growing literature on early childhood education has found that short-term impacts on test scores are inconsistent predictors of later-life impacts . . . Studies of teacher impacts on student outcomes show a similar pattern of results . . . It turns out that teacher impacts on test scores are almost entirely uncorrelated with teacher impacts on student classroom behavior, attendance, truancy, and grades . . . The teachers who produce improvements in student behavior and noncognitive skills are not particularly likely to be the same teachers who improve test scores.


You would think this disconnect would prompt plenty of furrowed brows and set off lots of alarm bells. It hasn’t. And yet the phenomenon that Hitt et al. note isn’t all that surprising if we think about it. After all, test scores may go up for many reasons. Here are a few of them:

• Students may be learning more reading and math and the tests are simply picking that up. All good.

• Teachers may be shifting time and energy from untested subjects and activities (like history or Spanish) to the tested ones (like reading and math). If this is happening, scores can go up without students actually learning any more.

• Teachers may be learning what gets tested and focusing on that. In this case, they’re just teaching students more of what shows up on the test—again, this means that scores can go up without students learning any more.

• Schools may be focusing on test preparation, so that students do better on the test even as they spend less time learning content—meaning scores may go up while actual learning goes down.

• Scores may be manipulated in various ways, via techniques as problematic as cheating or as mundane as starting the school year earlier. Such strategies can yield higher test scores without telling us anything about whether students actually learned more than they used to.

It matters which of these forces are driving rising scores. To say this is not to deny the value of testing. Indeed, this observation is 100% consistent with a healthy emphasis on the “bottom line” of school improvement. After all, results are what matters.

But that presumes that the results mean what we think they do. Consider: If it turned out that an admired pediatrician was seeing more patients because she’d stopped running certain tests and was shortchanging preventive care, you might have second thoughts about her performance. That’s because it matters how she improved her stats. If it turned out that an automaker was boosting its profitability by using dirt-cheap, unsafe components, savvy investors would run for the hills—because those short-term gains will be turning into long-term headaches. In both cases, observers should note that the “improvements” were phantasms, ploys to look good without actually moving the bottom line.

That’s the point. Test scores can convey valuable information. Some tests, such as the National Assessment of Education Progress (NAEP), are more trustworthy than others. The NAEP, for instance, is less problematic because it’s administered with more safeguards and isn’t used to judge schools or teachers (which means they have less cause to try to teach to it). But the NAEP isn’t administered every year and doesn’t produce results for individual schools. Meanwhile, the annual state tests that we rely on when it comes to judging schools are susceptible to all the problems flagged above.

This makes the question of why reading and math scores change one that deserves careful, critical scrutiny. Absent that kind of audit, parents and communities can’t really know whether higher test scores mean that schools are getting better—or whether they’re just pretending to do so."

"Since No Child Left Behind first rumbled onto the scene, the use of a Big Standardized Test to drive accountability and measure success has been a fundamental piece of education reform. But recently, some education reform stalwarts are beginning to express doubts.

There are plenty of reasons to doubt the validity of the Big Standardized Test, be it PARCC or SBA or whatever your state is using these days. After almost two decades of its use, we've raised an entire generation of students around the notion of test-based accountability, and yet the fruits of that seem.... well, elusive. Where are the waves of students now arriving on college campuses super-prepared? Where are the businesses proclaiming that today's grads are the most awesome in history? Where is the increase in citizens with great-paying jobs? Where are any visible signs that the test-based accountability system has worked?

Two years ago Jay Greene (no relation), head of the Department of Education Reform at the University of Arkansas, was writing about the disconnect in test scores-- if test scores were going up, wasn't that supposed to improve "life outcomes." Wasn't the whole argument that getting students to raise test scores would be indicative of better prospects in life? After all, part of the argument behind education reform has been that a better education was the key to a better economic future, both for individuals and for the country. Greene looked at the research and concluded that there was no evidence of a link between a better test score and a better life.

Here on Forbes.com this week, contributor Frederick Hess (director of education policy studies at the American Enterprise Institute, a right-tilted thinky tank) expressed some doubts as well. AEI has always supported the ed reform cause, but Hess has often shown a willingness to follow where the evidence leads, even if that means challenging reform orthodoxy. He cites yet another study that shows a disconnect between a student's test scores and her future. In fact, the research shows that programs that improve "attainment" don't raise test scores, and programs that raise test scores don't affect "attainment."

Test scores can be raised with several techniques, and most of those techniques have nothing to do with providing students with a better education. Drill the test prep. Take at-risk students out of electives and make them take test-related courses instead. And have teachers learn, over the years, how to teach more directly to the test. But do you want higher test scores or better education? Because those are two unrelated things.

The end result is that the test scores do not tell you what they claim they tell you. They are less like actionable data and more like really expensive noise.

Hess and Greene represent a small but growing portion of the reform community; for most, the Big Standardized Test data is God. For others, the revenue stream generated by the tests, the pre-tests, the test prep materials, and the huge mountains of data being mined-- those will be nearly impossible to walk away from.

But there is one critical lesson that ed reform testing apostates should keep in mind. The idea that the Big Standardized Test does not measure what it claims to measure, the idea that it actually does damage to schools, the idea that it simply isn't what it claims to be-- while these ideas are presented as new notions for ed reformers, classroom teachers have been raising these concerns for about twenty years.

Teachers have said, repeatedly, that the tests don't measure what they claim to measure, and that the educational process in schools is being narrowed and weakened in order to focus on testing. Teachers have said, repeatedly, that the Big Standardized Tests are a waste of time and money and not helping students get an education. Teachers have been saying it over and over and over again. In return teachers have been told, "You are just afraid of accountability" and "These tests will finally keep you honest."

After twenty years, folks are starting to figure out that teachers were actually correct. The Big Standardized Test is not helping, not working, and not measuring what it claims to measure. Teachers should probably not hold their collective breath waiting for an apology, though it is the generation of students subjected to test-centered schooling that deserve an apology. In the meantime, if ed reform thought leader policy wonk mavens learn one thing, let it be this-- the next time you propose an Awesome idea for fixing schools and a whole bunch of professional educators tell you why your idea is not great, listen to them."

An Essex man who had £12,500 worth of bikes stolen from his home has warned cyclists to be careful of what they post on social media. Adam Jones believes thieves may try and identify riders who record fast times on Strava segments and he expressed regret that he had not been more cautious regarding the information he shared online.

The Echo (link is external) reports that on Wednesday night, five bikes were taken from Jones’ Barling Magna home: a Specialized Tarmac, valued at £1,500; a Bianchi Infinito CV, worth about £4,000; a Dolan Scala time trial bike, worth about £2,000; a Bianchi Oltre XR2 worth £2,500; and a BMC Road Machine, valued at about £2,700.

might be a useful tool in my never ending pursuit of functional metrics for myself as a human

The vehicles were officially designed by Italian company Pininfarina, which collaborated with VinFast. Before the design process started in earnest, VinFast took 20 sketches from four Italian car design houses and let the public vote on their favorite in a nationwide poll that attracted 62,000 people. The images below are the ultimate fruits of that early voting on what styling direction VinFast should take.

Owners of Vizio Smart TVs could soon see a message inviting them to take legal action against the maker.
Vizio, one of the leading TV brands in the US, was caught gathering large amounts of user data in 2015, which was then passed on to advertisers. It settled with the US trade regulator for $2.2 million, but still faces a class action lawsuit involving potentially millions of customers.
The company could now be forced to inform potentially affected owners directly – via a pop-up message on-screen – reports the BBC.

The biggest challenge for UK railways is keeping up with demand. Since privatisation in the 1990s, rail use has doubled and it’s expected to do so again in the next ten years. There are currently more than 1.7 billion rail journeys every year in the UK.

Euan McLeod, head of infrastructure for HSBC UK Commercial Banking, says: “Rail operators and rolling stock financing vehicles are successfully supporting the modernisation of the system, including investment in stations, trains and better services. More of this should be encouraged, and even extended into tracks and signalling where investment is needed. There is real appetite from private finance to support rail investment in the UK and at record low interest rates too.” The question is whether resources are being deployed in the most effective way.

Planning & #measurement of the intended position of the #acetabular #component in the supine position before total…

In Virginia, the governor’s office has launched an innovative effort to bring a geospatial angle to issues of food availability and healthy eating.
The Virginia Food Access Network (VFAN) uses maps to tell “data stories” around such issues as childhood hunger, access to nutrition and availability of local produce. The data comes from government sources, non-profit organisations, academia and elsewhere.
“They had all this information at the county level, but people’s eyes would glaze over when they saw it on a spreadsheet,” says Rob Rose. As director of the Center for Geospatial Analysis at the College of William and Mary in Williamsburg, Virginia, he helped to develop the GIS component of the government’s VFAN site.

it is very difficult to properly benchmark complicated systems.