“Long experience has taught me this about the status of mankind with regard to matters requiring thought: the less people know and understand about them, the more positively they attempt to argue concerning them, while on the other hand to know and understand a multitude of things renders men cautious in passing judgment upon anything new.”
― Galileo Galilei

One of my more pleasant discoveries since I began writing this blog is the willingness of people in the education profession to share their knowledge. I know that should be self explanatory, after al,l most are teachers at heart, but stil,l people reach a certain stratosphere of success where it becomes easier to limit your engagements. Over the years I’ve been very priviledged to talk with and learn from some of the smartest people in the field. An access I certainly don’t take lightly.

Since last Thursday’s Chicken Little announcement by Commissioner Penny Schwinn that the state could see student proficency scores drop by 50% in ELA, and 65% in math, there have been a lot of questions about how those numbers were derived at. In a follow-up email sent to state superintendents by the commissioner last Friday, she cited a recently completed study by CREDO as the source of those predictions.

The Center for Research on Education Outcomes (CREDO) is housed at Standford University and has been doing extensive research in the field of public education for over two decades. Wether they have an intentional bent or not, over the years CREDO has become the go to source of data for those making the case of charter schools over public schools.

Founded by Margaret (Macke) Raymond and Eric Hanushek, both of whom had experience in the realm of political science but no classroom experience. Since inception their research has been cause for much public debate over education policy. Given her history, and his procolivity, it’s not surprising that Penny Schwinn and Governor Lee chose a CREDO study in which to fuel their individual agendas.

Despite Schwinn’s follow up email to state superintendents, there still remains a lot of questions around exactly how loss of learning numbers were translated into predictions on proficency. To most it was a rather large leap, one without a clear path. Since the TNDOE was not very forthcoming with any explanation, I decided to call Macke Raymond and see if she could provide anymore clarity.

Yesterday I sent her an email asking if she could possibly find time in which to answer some questions I had. She promptly responded with an email with her cell number and a window of time in which she would be available today.

I must admit that I approached our conversation with some trepidation. Assessment research and the analyzation of data is an extremely complex science, that has a certain level of art to it. To the person not fluent in its applications and language, it can quickly becoome confusing. Make no mistake, I’ve been the benificary of many a conversation with those fluent in the science, but I myslf am not.

At the earliest available time I called Ms. Raymond, who answered on the first ring. Any trepidation I initially felt, instantly receded as I discovered a warm researcher who was willing to patiently explain her research. It was a confersation I very much enjoyed.

The study that Ms Schwinn referenced is part of a much larger study that CREDO will be releasing at the end of the week. That larger study is similar to the one provided to the TNDOE but includes 19 other states as well. It will provide meta data that will give considerably more insight than that given in the Tennessee study. It will also paint a little different picture than the one painted by Commissioner Schwinn.

The Tennessee study was done with historical data that was available to CREDO due to a decade long partnership between the researchers and the state. No Tennessee school districts provided data from any benchmark tests done since schools reopened. The CREDO results were all predictions made by using sophisticated algorythems and past data.

I raised the question around converting the data back to scale in order to predict proficency scores. Raymond indicated that additional data wasn’t publically available due to it being based on individual student scores and as such was subject to FERPA protections. She did indicate that CREDO did not calculate predicted proficency scores and assumed those calculations were done by the department itself, since they had access to cut scores.

She also explained that their modeling was based on 3 different levels of schooling provided, starting with no instruction provided and then adding from there. When I raised the question around the assumption that kids lost 3 months of instructional time by being out of school during the Spring, despite that being testing time in Tennessee, she countered that there was still aproximately 50% of instruction being delivered.

In talking to her I got the sense that her research was less about whether kids were physically in school or not and more about how COVID-19 is shining a light on the vast inequities that already exist within our school systems. We were both in agreement that if you were fortunate to have strong supports and attend a strong school in person you were probably doing well with the remote learning provided by your school. For those in less fortunate situations and in weaker schools, the converse held true.

Prior to COVID-19 there was an argument that there were no good or bad schools, and that performance was overly impacted by demographics and resources. I think COVID has punched a whole in that arguement. While both hold considerable sway over outcomes, building leaders also carry a considerable amount of weight and shouldn’t be overlooked. The shift to remote learning has made it impossible to overlook the disparitys between schools.

Renke expressed a hope that education leaders would take the data made availble by CREDO and use it to recognize that it was going to take a combination of community, additional resources, and hard work in order to mitigate the inequities in our system. It’s a hope that I actually share with her.

I did raise the question around Ms Schwinns assertion that all kids were suffering from elevated learning loss as a result of school closures. As noted by Chalkbeat, “Schwinn said data collected by the state shows that learning loss is affecting students consistently across Tennessee, whether they live in urban, suburban, or rural communities.” It was the only time Ms Raymond bristled during our conversation. She countered that the CREDO study absolutely did not say that. In their study they acknowledged that some students have actually made gains during this time of remote learning. But it also showed that many of those who can ill afford to lose ground, are losing even more.

In light of this, I wish the argument would stop being about returning to in-person learning vs continueing virtually, and instead focus on how to create more supports, no matter how instruction is delivered. I’ve talked about using community partnerships in order to provide learning centers and pods to those kids who’s parents can not afford it. Expanded tutoring services would be a welcome addition. Increased focus on building up substitute teacher numbers is a neccesity.

When I was running for school board, I promoted the idea of finding a way to provide health insurance for substitutes based on the number of days a quarte they worked. It’s imperative that districts develop plans that allow substitute teachers to attend district run professional development.

So instead of racing to shove kids back into school buildings before we can ensure their safety, I would argue that we’d be better served taking the CREDO data and using it find ways to deliver supports to teachers and students regardless of how instruction is delivered.

There are very supports that we could create at this time that would not be beneficial when kids transition back to live instruction in larger numbers. Even then, remote learning will continue to be part of the equation. Those are envisioning some kind of transition back to an idylic version of bricks and motar schooling, might just as well wish for a return to the one room school house. Neither is likely to transpire.

That still leaves us with the question of, how did Commissioner Schwinn arrive at the her predicted efficenciy rates? Per ChalkbeatTN,

Pre-pandemic test data analyzed by national researchers — not recent back-to-school test results from Tennessee students — was the basis for state projections this week that proficiency rates will drop by 50% or more for third-grade reading and math due to schooling disruptions during the pandemic.

After talking to Macke Raymond today, that doesn’t seem to be accurate. Per our conversation, CREDO never computed proficency rates. They left that to the state.

A new article by Vivian Jones for Center Square seems to indicate a different source for the predictions. Jones writes,

Analysts at the education department incorporated data from beginning-of-year student checkpoints that school districts voluntarily submitted this year. Results from the checkpoint tests show students’ actual learning loss is consistent with the department’s projections of 50 to 65 percent loss, Victoria Robinson, a spokesperson for the department, confirmed to The Center Square. Beginning-of-year testing is ongoing. 

So let’s think about this for a second. The headlines for the last several days has been centered around the use of pre-pandemic data to make decisions and now department spokes people are shifting to a narrative that it’s current benchmarks that provided the data. Benchmarks that have only been taken by 3% of the students in the entire state. Can you say. “small sample size”, coupled with, “shifting sands”?

As the departments story continues to unravel, I think it’s safe to say that we’ve arrived at a point where the department owes some concrete explanation. It’s time for Chief Strategy Officer Mike Hardy to step forward. It was his office that took the two data sets and linked them in order to predict proficency rates, just tell us how you did it.

As a by product of this lack of clarity, many question whether Hardy and his team are capable of doing the calculations that are required to diagnose the needs of the states students. A misdiagnose based on faulty analysis, has the potential to rob students of important opportunities. It can lead to uneccesary remeadiation and and investment in unneeded programs. It can also cause a loss of faith in not only local superintendents but also the principals and teachers that are working so hard to meet the needs of Tennessee children.

This is where the high attrition rates at the department also come into play. People leaving may portray Schwinn as a dysfuncional leader uncapable of inspiring the troops, but more importantly high attrition rates rob the department of vital resources. Lack of resources hinder the TNDOE from producing meaningful results. There are a limited number of people who posess the unique skill set required for high level department jobs. The idea that you can discard one and just arbitrarly pick up another is an incorrect assumption. What inavariably happens is you pick up an employee with an inferior skill set and as a result the work suffers.

Want to prove me wrong? Want to prove that predictions are in fact accurate and based on reliable data? Get Grady to explain how he arrived at the predictions that the commissioner delivered. That’ll make things crystal clear.

For the record, the Governor’s response has been no better than Schwinn’s. According to News 5 out of Memphis, when asked by a East Temnnessee reporter about the questions around Schwinn’s statements he answered,

“We’re looking for as much information as possible, as early as possible to begin to understand what learning loss looks like so that we can make adjustments so that we can have interventions so that we can minimize the amount of loss that (students) have, and give them a greater opportunity for a future,”

Um…no…your Education Commissioners statements don’t indicate that you are looking for anything, but rather that you’ve already found it and you think there is only one solution – a return to in-person schooling.

Let me leave you with a little ancedote from my conversation with Macke Raymond today.

We’d been talking for a bit when she made the following observation, “If I’m not mistaken, you also referenced the economic study included in the press conference.”

“Yea”, I responded, “That one’s another issue. First off, it’s a working paper…”

“Oh it’s not a working paper, it’s finished.”

“Well it still attempts to predict something I don’t believe can be predicted. Earnings over a life time is very much a work in progress, and that’s coming from someone who went to school for 5 years but left without a degree. Made a bunch of money, lost a bunch of money. Became an addict and lost several years of earnings. Got sober and made some more money. Hit by a pandemic and is once again struggling. Nothing could have predicted that trajectory.”

“So I should tell my husband that you take exception to his paper?”

I burst out laughing and she joined me. “Well I do have a propensity to put my foot in my mouth,” I replied, “But yea, if that’s your husband…I guess you could tell him that.”

While on the surface that is an amusing story, it also sheds a light on some of the shading that the Governor is engaged in. The press release makes it sound as if Lee has employed two seperate entities in order to draw his predictions of glum. Under closer scrutiny it’s revealed that they are derived from the same household. I am anxious to find out what even closer scrutiny reveals. I look forward to release of the data in the much longer CREDO study.

I would especially like to thank Macke Raymond. You’ve given me a lot to think about, and hopefully today won’t be the last time we talk. If only it was so easy to get information as readily from the Tennessee Department of Education.

That’s it for today. But I’d be remiss if I didn’t say thank you, teachers and administrators, for everything you do.

If you’ve got time and are looking for a smile, check out the Dad Gone Wild Facebook page, where we work to accentuate the positive.

If you’ve got something you’d like me to highlight and share, send it on to Norinrad10@yahoo.com. Any wisdom or criticism you’d like to share is always welcome.

Categories: Education