“No matter what happens in the world, however brutal or dystopian a thing, not all is lost if there are people out there risking themselves to document it. Little sparks cause fires, too.”
― Swimming in the Dark
The field of student assessments is a complex one. Its a field heavily reliant on high level math, complex algorythm, and a deep understanding of subject matter. Used properly, assessments can give great insight into student progress, where the gaps in learning fall, and effective practices. It can give better guidence on the deployment of resources and help shape policy in order to benefit kids, and ultimately society.
Unfortunately the flip side can be particularly damaging. It can lead to the prescription of unneccessary and harmful learning plans. Resources being squandered due to mis-perceptions. Mis-interpretations and mis-use can result in teachers and schools being painted in an inaccurate light, giving the public the misperception that they are failing kids, and as a result, hampering educator efforts.
For these reasons and more, great care needs to be taken in intrepetating data and creating a narrative around those intrepations. Two things that Tennessee’s Commissioner of Education Penny Schwinn failed to do when she stood in front of reporters and declared that data showed that due to the pandemic, and specifically a lack of students attending brick and mortar schools, proficency scores in literacy were expected to fall by 50% and math by 60%. Needless to say, those numbers caught people’s attention.
It also caught people by surprise. Just the day before Schwinn had appeared before the House Education Committee, where she testified for over 90 minutes about schools and the pandemic. During that time she made reference to sharing data about learning loss, but ultimately never presented the data.
It’s interesting that during per time she managed to talk about her DC appearances in front of Congress and the department’s partnership with PBS, but somehow data showing a learning collape by Tennessee students managed to fall to the bottom of the list of important subjects to cover.
Midway, through her presentation Chairman Mark White attempted to give her cover from criticism by legislators by remarking how difficult the job had been on her predeccssors McQueen and Huffman. He even asked for input on what the elected body could be doing to make the TNDOE’s job easier. Maybe it’s just me, but wouldn’t that have been a good time to make legislators aware of the uncomming predicted crash of proficiency scores? Maybe ask for a few more resources to counter the crash? After all she had their undivided attention and they seemed very open to being strong partners. But the commissioner chose to utter nary a word.
The next day, she held a conference call with the state’s individual district superintendents. During her presentation on the previous day, she had bragged to legislators about holding these calls 3 times a week and how they served to strengthen relationships. One might think that since these Superintendents were the one who would have to deal with the fall out from such a devestating announcement, as well as being the ones that would have to fix things on the local level, Ms. Schwinn might have wanted to maybe give them a heads up. Not a word was uttered during that conference call.
Superintendents and legislators found out at the same time as the rest of us, at the governnor’s press conference. The announcement predictably produced an instant hue and cry.
Media outlets quickly took up the clairion call of pending doom and gloom. Most choosing to go with the headline on the TNDOE’s press release – Tennessee Releases Data Showing Significant Learning Loss Among K-12 Students. In their rush to sound the alarm, few noted that the state did not release any data, merely statements from Department leaders. But superintendents certainly noticed.
Their response was equally fast and furious. So much so that, Commissioner Schwinn was forced to send them an attempt-to-clarify email and schedule an afternoon ZOOM call with Dale Lynch and TOSS. This is expecially significant because Lynch has practically thrown his back out carrying water for Schwinn since her arrival. A screenshot of particpants gave a pretty clear indication of thier state of mind,
Quick side note, over her brief tenure, Schwinn has had to write so many of these so-called “clarification letters that maybe it’s time to create a seperate department within the department that focuses on doing nothing but writing apology letters. I’d argue that there is enough work to keep a couple staffers busy.
In any case, neither the letter, nor the apology served to assuage the concerns of district leaders. Per Joey Hassell, superintendent of schools in Haywood County, near Memphis,
“The e-mail did provide context for the data shared in the release on Wednesday; however, it did nothing to reassure me that we could trust the department or expect a public apology regarding an ill advised press release.”
In her email to Superintendents Schwinn cited the following as a source for her prediction,
Nationally-recognized research and assessment organizations, NWEA and CREDO (who partnered with NWEA) reported the estimated projections of learning lossnationally andin Tennessee.NWEAreleased an April 2020 study to estimate the effects of time out of school on learning loss. This nationwide study can be found here: https://www.nwea.org/content/uploads/2020/05/Collaborative-Brief_Covid19-Slide-APR20.pdfand in a sample news articlehere.
Ok…but there are a couple of caveats that belong here. The NWEA study linked was done in April and took into little account what students were doing virtually. It also focused on learning loss, which does not translate into proficency rates. Just because students lose 50% of what they learned, does not translate into half of them scoring less proficent on TNReady. It’s important to note that the time NWEA was counting as lost instructional time was during the time most districts would be engaged in their annualized standardized testing. So at most you could equate student learning loss to two months.
Now the way the commisioner presented things, the assumption would be that if a student had X learning loss for those 2 months, than they would have X2 for four months and you could extrapolate it on out. That’s not the way it works, the loss is one of dimensioning terms. Over time the loss slows until eventually it stops.
Also keep in mind that the “summer slide” does not effect all kids equally. It is not uncommon to see higher achieving kids and those from more affluent socio-economic backgrounds actually show growth over the summer months because they are going to trips, mueseums, engaging in more free reading, and various other activities that augment learning. Summer slide, as to her credit Schwinn acknowledges, effects kids from impoverished homes more dramically. But she didn’t say at-risk kid’s proficiency rates were going to be dramically hit, she said all of Tennessee’s kids.
In her email to supereintendents, Schwinn included further explanation of the CREDO study(COVID-Sim PowerPoint TN – CREDO). To be perfectly clear, no where in this study does CREDO use current student data nor does it talk about proficency. As explained by an admiistrator well versed in assement practices, for the study they took prior year’s data to project what scores would be in spring 2020 if there were no closures. They tried five different models and actually end up using the third best one to make their projections. They project the distribution of score, sans closures and then convert those to standard scores, basically the mean of distribution is 0. Then they just apply their findings on learning days calculations to subtract .1 std deviation for closures for all students and extrapolate the learning loss from NWEA findings which differ by ventile. They never share their projections which is why it is impossible to translate back.It’s a mouthful.
Again no where in the report does CREDO say, “Math proficency rates will drop by 65% and Literacy by 50%.” Nowhere in the report does CREDO indicate that they are using current student measurements. A concise summary might be…CREDO made projections for spring 2020 just like SAS – the company that oversees TVAAS – does every year. Then they made the assumption no student learned post-covid in the spring. They used prior research on learning days and summer slide to project learning loss from their projections, which are all presented in terms of std deviations and never translated back to tnready scale scores or proficiency in the released materials. Thus claims by Ms. Schwinn remain unsubstantiated.
Equally important is the caveat that comes with the report.
Commissioner Schwinn has made vague references to Tennessee districts that have shared current data in order to produce a report, but has failed to name who those districts are and as of present nobody has been forthcoming. And again there is no reporrt produced with current data.
Over the summer NWEA did release a study that attempted to link TNReady and MAP that utilized data from 7 Tennessee districts, but it is unclear if those are also the districts Schwinn is referencing in regard to their data under current circumstances.
A couple quick things are worth noting with the linking study. In looking at the charts it would appear that 73% of 3rd graders in the country couldn’t score “On Track” on TNReady. That should probably be a part of our broader conversation around assessments. It says something about our students if in order to score “on track” on TNReady you have to outscore over 70% of the kids in the country.
Of course this study also comes with caveats,
Although the results show that MAP Growth scores can be used to accurately classify students as likely to be proficient on the TNReady tests, there is a notable limitation to how these results should be used and interpreted. TNReady and MAP Growth assessments are designed for different purposes and measure slightly different constructs even within the same content area. Therefore, scores on the two tests cannot be assumed to be interchangeable. MAP Growth may not be used as a substitute for the state tests and vice versa.
One last study to look at. In the press release accompanying Ms. Schwinns predictions is a link to another study, this one on potential economic consequences of a drop in student proficiency. Its credited to research from the Organisation for Economic Co-operation and Development, but again it comes with a caveat. First it’s not a study, it’s a “working paper” and one of the first sentences in the report states,
OECD Working Papers should not be reported as representing the official views of the OECD or of its member countries. The opinions expressed and arguments employed herein are those of the author(s). Working Papers describe preliminary results or research in progress by the author(s) and are published to stimulate discussion on a broad range of issues on which the OECD works.
For those keeping score at home, that’s three studies, none of which say what Penny Schwinn says they say. All with caveats saying, don’t use them the way the commissioner is trying to use them. Clearly what we have here, is a failure to communicate. Why?
Well my initial response would say the action is intentional and being used to drive the Governor’s desire to get students back in school and the commissioners intent to reshape the state literacy instruction and practices. Furthermore, it gives them both the opportunity to give legislators a big finger for questioning them.
I still think that they are sending a message to legislators, but I’m not sure it is intentional. To be honest, I don’t think they give a second thought to the desires and wishes of the General Assembly. It’s my argument that legislators cross the mind of Lee and Schwinn about as often as the concerns of a long suffering wife cross the mind of a philandering husband. Like the forementioned wife, legislator concerns are only relative when they can serve the needs of the governor and his chief educator.
To intentionally distrepect someone, you have to start with a basic interest in their thoughts. Something that the Lee administration clearly does not suffer from when it comes to the states other elected officials.
So if not vidictive, than what? The more I consider circumstances, the more I think that Hanlon’s razor can’t be dismissed – never attribute to malice that which is adequately explained by stupidity. Its often hard to discern, and at times forced by circumstances. But in this case, there was no urgency to comment.
A few years ago there was a problem with students accessing the TNReady tests. Then Commissioner McQueen offered the explanation that the testing system had been hacked. A charge that later turned out to be false and as a result was widely ridiculed.
In the defense of McQueen, she was in crisis mode and the company producing the test, Questar, put forth the explanation. In the heat of the moment she had little choice but to accept Questar explanation. For what it’s worth, Tennessee is no longer a client of Questar.
In this case. There was no emergency. This was a planned and scheduled press conference. Nobody was demanding any answers. There was no reason why the the information should not have been fully vetted before being presented. Something that clearly was not done.
In trying to assertain what exactly transpired, I would say that we should start with Michael Hardy. Hardy is the Chief Strategy Officer at the TNDOE. In that role he is charged with setting the vision and improving the quality of data for schools across all of Tennessee by managing Assessments, Accountability, Data Use, Data Governance, and department-wide strategy. Before coming to the DOE in June of 2019, Hardy spent 8 plus years with IDEA Public Schools in Texas. He’s also a former TFA corp member.
Wait a minute, who else spent some time employed by IDEA? I know there is someone else I know. Give me a second…oh…yea…I’ll give you a hint, if you go by the Schwinn house, you’ll find his initials on some monogramed towels.
Keep in mind that IDEA Public Schools is not an organization known for it’s ability to handle figures. In fact, if you think about it, none of the TNDOE’s Texas transplants offer a strong argument for compentcy. You have Robert Lundeen and his inability to monitor special ed vouchers. Katie Houghtlin who can’t figure out how to treat people she supervises. Rebecca Shah who only signed on for a year, Now Hardy. Hell Schwinn cost the state of Texas over $5 million over her incompentence. All of this would be quite comical if it wasn’t so serious, and it’s very serious.
The CREDO study cited Schwinn lists the multitude of uses for assessments both on the state and local level. It is absolutely imperative that the data is translated and applied correctly and with fidlity. That’s non-debatable.
Furthermore, the data put forth by the commissioner is faithfully reported by the media as being factual. In this case it was not factual, but it still served to undermine the herculean work of the states professional educators. In her email to superindents Schwinn attempted to walk back that undermining,
This data does not reflect the exceptional planning and hard work that has been done to open schools so well. This is also why it will continue to be necessary to prioritize a focus and commitment to our schools. Teachers, principals, staff and superintendents have made heroic efforts and will need our continued and increased support as the academic needs of students have increased. This data makes the case for our continued collective focus and urgency on education in the state.
Every district in the state is unique and different, as is every student. I expect that this representative state level data does not reflect the exact proficiency or learning loss reflective of every individual district, in the same way that statewide averages on TCAP do not reflect the specific performance of every district.
I remain confident that the difficult choices your districts are making and the time and effort that is being put into this school year will meet the needs of our students and give us continued reason to laude and celebrate the work our Tennessee schools have done on behalf of our children.
It was a disclaimer that would have been better served had it been offered before the Governor’s press conference, As it now stands, peace sells but no one is buying.
This is a story that provides much more fodder for conversation than I can cover here today. We still have to have a broader conversation around the sharing of district benchmark data. Who’s entitled to it? What’s it proper usage? Are parents being made fully aware of the potential usages?
There still needs to be a whole lot more conversation around the $40 million dollars in grants for literacy mentioned by Commissioner Schwinn last week. What are they designated for? How will the monies be applied? And how much oversight on their usage does the General Assembly have?
Not to mention what are the factual determinations being made out of the data currently being collected by LEA’s?
Hopefully, the state’s superintendents are letting this latest mis-step by Ms. Schwinn go by without reprecussions. It should not be comsidered a mere slip by anybody and hopefully they continue to talk with their state representatives in order to get a clear picture of exactly what’s going on when it comes to educationg the children of Tennessee.
That’s it for today. But I’d be remiss if I didn’t say thank you, teachers and administrators, for everything you do.
If you’ve got time and are looking for a smile, check out the Dad Gone Wild Facebook page, where we work to accentuate the positive.
If you’ve got something you’d like me to highlight and share, send it on to Norinrad10@yahoo.com. Any wisdom or criticism you’d like to share is always welcome.
A huge shout out to all of you who’ve lent your financial support. I am eternally grateful for your generosity. It allows me to keep doing what I do and without you, I would have been forced to quit long ago. It is truly appreciated and keeps the bill collectors happy.
If you so desire to join the rank of donors, you can still head over to Patreon and help a brother out. Or you can hit up my Venmo account which is Thomas-Weber-10. I don’t need much – even $5 would help – but if you think what I do has value, a little help is always greatly appreciated, especially this time of year when my contracted work is a little slow. Not begging, just saying.
Don’t forget, if you have student-written blog posts you’d like to see reach a wider audience…send them on. I’d love the opportunity to share them.
Categories: Education
Dad Gone Wild 
Leave a Reply