I often get emailed things that just don’t seem to add up on initial inspection and require a little investigation. This past week, I received two very interesting emails. After diving into these queries, I decided that they were worthy of sharing.
The first email came from a teacher who received the following message from the State in regards to her TVAAS score:
We wanted to share an update with you regarding your individual TVAAS data.
The department has processed about 1.5 million records to generate individual TVAAS scores for nearly 19,000 educators based on the assessment results from over 1.9 million student tests in grades 2-8 and high school. During the review process with districts, we found that a small number of educators did not have all of their teacher-student claiming linkage records fully processed in data files released in early September. All linkage data that was captured in EdTools directly was fully incorporated as expected. However, due to a coding error in their software, our data processing vendor, RANDA Solutions, did not fully apply the linkage information that districts provided in supplemental Excel files over the summer. As a result, we are working with Randa to ensure that this additional data is included in final TVAAS processing.
You have been identified as an educator with some linkage data submitted via an Excel file that was not fully processed. This means after our statistical analysis vendor, SAS, receives these additional linkage records, you may receive an individual TVAAS score if you did not originally have one, or your individual TVAAS score may be revised to reflect all the students you identified in the teacherstudent claiming process.
Your district’s and school’s TVAAS scores are not affected by this situation given that all students are included in these metrics, regardless of which teacher is linked to them, so no other part of your evaluation composite would change. Moreover, only those teachers with this additional linkage data in Excel files are impacted, so the vast majority of your colleagues across the state have their final individual TVAAS composites, which are inclusive of all student data.
We expect to share your final growth score and overall level of effectiveness later this year. While we do not have more specific timing to share right now, we are expediting this process with our vendors to get you accurate feedback. We will follow-up with more detailed information in the next couple of weeks. Also, as announced to districts earlier this month, the department and your districts will be using new systems and processes this year that will ensure that this type of oversight does not happen again.
Thank you for your patience as we work to share complete and accurate feedback for you. We deeply value each Tennessee educator and apologize for this delay in providing your final TVAAS results. Please contact our office via the email address below if you have any questions.
Respectfully, Office of Assessment Logistics Tennessee Department of Education firstname.lastname@example.org
I’m going to be the first to admit here, I have no idea what they are talking about in this letter. Neither does the teacher in question. I will say few things make you feel valued like a form letter filled with mumbo jumbo that’s tied to your job evaluation.
I could pick up the phone and call the lovely Sara Gast over at the TNDOE, and I’m sure she would patiently give me an answer. Now I can’t promise that after talking with her I would have any more clarity, but that’s through no fault of Ms. Gast. The problem is inherent in the process itself.
Quick survey here: raise your hand if you would be okay with a job evaluation that you couldn’t explain to a new employee or a friend? Any hands? I don’t see any. Yet that is the evaluation process we are using on teachers.
I’ve been doing a lot of thinking and questioning on how we are computing TVAAS this year. The challenges are quite unique, given that they started with one test, followed by a year of no data, and then a new test that is completely different the following year. Out of these data bases, a growth score is somehow supposed to be calculated.
So best I can tell, what’s happening is each teacher has a group of kids who took Test A. Then a bunch of those kids took Test B. All those kids, based on past scores, were predicted to score about the same on Test B. If they did, you earn a 3 for your growth score. If they scored higher than they were predicted, you earn a 5, and if they score less, you earn a lowly 1.
Now according to this letter, apparently all of a teacher’s students couldn’t be matched between tests. As a result, there are not enough students accounted for in order to calculate a growth score. Or it could just be that a file wasn’t transmitted. Obviously I’m guessing here, but a veteran teacher left a comment explaining to me that after the tests have been scanned, but before they are scored, each teacher has a list of their students who took the test. To make sure the lists of students “assigned” to a teacher are correct, teachers have to perform a “linking” process, which is where a teacher logs on to a state testing site, reviews the rosters connected to them, and essentially says “yes, those are my students.” In some cases, they indicate if a student was only in class half the year or if they had excessive absences, in which case those students won’t count as much in the overall TVAAS formula. Now the State’s supposition is that it must be these linking files that got messed up in transit.
I know the process is a lot more scientific than what I’m describing and it uses lots of mathematical formulas, words like “cohort” and “indicators,” along with a little smoke and a lot of mirrors. The problem is, at the end of the day, you have an evaluation system that you have to be a statistician in order to understand how the State arrives at a score. That would be fine if that score didn’t carry so much weight, but it carries an incredible amount of weight for educators, schools, and even communities.
That growth score influences a teacher’s future opportunities and employment status, school funding and governance, and district strategies and priorities. All of which, in turn, influence property values and economic growth for communities. Shouldn’t a formula with so much potential impact be a little less opaque than taking a dead cat out to the graveyard at midnight, twirling it around 10 times, releasing the cat while chanting “I am a good teacher” 10 times over, and then waiting to divine the results based on whichever way the wind blows?
In 2014, the Tennessee Education Association sued the State over the use of TVASS scores and lost. However, in light of recent developments and the continued lack of clarity involved in the process, I can’t help but think someone will try again. Teachers in Tennessee deserve an evaluation system that is understandable by everyday people and not one that is reliant on complex verbiage that leads to continual corrections and clarifications.
SHARKS VS. THE JETS
The other email I received last week was from the mother of a Hume-Fogg High School girls soccer player. As many of you know, Hume-Fogg is one of two academic magnet high schools in Nashville – MLK being the other. Both schools are widely recognized as being two of the top schools in Tennessee and in the upper echelon of high schools in the country. It’s safe to say these schools are made up of the best and brightest in the country.
What you might not know, and admittedly I didn’t, is that they also have two of the best girls soccer teams in the county. Last week they were scheduled to square off for the district championship. Not surprisingly, these two teams share a passionate rivalry. Such that at a semi-final game earlier in the week between the two, their respective student bodies ended up squared off in the parking lot for a potential rumble after the game.
How ugly things actually got I don’t know. Reports vary from “no punches thrown” to “it got pretty hairy.” Whatever the case, it attracted the attention of both school principals, who got together and declared that no students would be permitted to attend the district finals game.
In all fairness to the principals, they had issued repeated warning to students on the potential ramifications of their behavior. But I would argue that the punishment was a little harsh. This is the championship game, after all. High school is where we all create memories that stay with us for the rest of our lives.
How many of us can recall with clarity that calculus test you aced senior year? But you probably will remember what it was like to win that championship in front of your peers and family. That validation for all the hard work and sacrifice given to chasing an athletic goal, on top of all the hard work and sacrifice in pursuit of academic excellence is something that will color the rest of a person’s life. It seems to me that an alternate solution could have been found.
The parent who sent me the note had a younger child, in addition to the team member, who wanted to see their sister compete for the championship. Obviously the edict of no student attendance was a big concern for them. I put the parent in touch with MNPS’s Executive Officer of Student Services Tony Majors. Majors, per usual, did his best to help the parent navigate the situation. The edict stood for the championship game, but has since been lifted for future sporting events.
Oh… who won the game? Hume-Fogg did. Congratulations to the 2017 District girls soccer champions.
A few national items I’d like to call your attention to.
I often talk about how much sacrifice we demand from our teachers. Education blogger Othmar’s Trombone wrote this week about how teaching has become a profession dependent on self-sacrifice and martyrdom – TEACHING: IF YOU AREN’T DEAD YET, YOU AREN’T DOING IT WELL ENOUGH.
It’s no secret that I am not a huge proponent of STEAM instruction. One of the main reasons being that it relegates the arts to a support status. They are perceived as being valuable only through their relationship to other more academic disciplines. Jay Greene writes this week on that very subject – Arts Integration Is a Sucker’s Game.
While Nashville drags its feet over lead in schools’ drinking water, in Philadelphia, parents are going even further and looking at the lead levels in the dirt of local parks and playgrounds. It’s all quite concerning. Rumor has it there will be another report tonight on lead in Nashville schools’ drinking water by Phil Williams on Channel 5. Let’s see if anyone is paying attention.
Update: Phil Williams did do another story last and it is another eye opener. This one illustrates just who is getting hurt by the high levels in MNPS school’s drinking water. According to the report schools with predominately middle class students have PTO’s that are capable of providing filtration systems for their schools. Higher poverty schools do not have PTO’s that are capable of such an investment. There is one quote by MNPS that really sticks in my craw as it’s thrown out there as a defense.
A Metro Schools spokesperson told NewsChannel 5 Investigates that it could cost up to $2 million to install filtration systems on water fountains throughout the district and up to a million dollars a year just to replace filters.
So? That doesn’t seem to0 steep a price to pay in order to prevent children from being exposed to the health risks of lead in the drinking water.
Once again our school board is silent. With elections for half the seats coming up next year it would be interesting to get their take on camera.
If you’ve been keeping up with the data wars between the TNDOE, Memphis, and Nashville, here’s a story you might want to put on your radar – Data breach exposes hundreds of Palo Alto High records.
Infinite Campus is the tool that MNPS uses to provide parents information about their kid’s academic performance. It’s a national company that is also utilized by the Palo Alto school district. Last week Palo Alto high school records were leaked. The leak from the Palo Alto Unified School District’s Infinite Campus electronic locker system included names, student identification numbers, and weighted grade-point averages for Palo Alto High sophomores, juniors, and seniors.
I think I have a bit of a reputation for being overly critical of MNPS leadership. One that may or may not be deserved. I will always argue that you are hardest on the ones you love the most. In creating each week’s poll questions, my goal is to gauge the general opinions of readers as well as looking to affirm that I am covering things in an accurate manner. I never create questions for the sole purpose of cementing negative opinions.
The underlying goal of the questions is always to foster conversations. So while I suspect questions will get negative responses, I still hope that the opposite occurs. This week’s questions fell into that vein. I knew what I thought the answers would be, but hoped for positive news. While I wasn’t fully countered on my assumptions, I do believe that there are positive threads woven into the answers given. Let’s look at the responses.
The first question asked you to grade the district for the just-completed first quarter. Fifty-four percent of you responded that the district was failing to meet expectations. That certainly is not good news. Thirty-nine percent of you, though, responded in a manner that indicated that the district was either meeting expectations or moving in that direction.
I translate that to indicate that while there is still not widespread buy-in to district leadership, the people doing the day-to-day work are finding ways to overcome the obstacles presented. They are still finding a way to make things happen, instead of just throwing their hands up and capitulating. It says a lot about the dedication of the teachers and administrators in MNPS. Hopefully, district leadership will take notice and begin to remove some of those obstacles and free up our talented educators to get more done. Here are the write-in votes:
|Bending under the weight of all this homework and literacy plan||1|
|Please fire Dr. Felder-she’s ruining our system||1|
|Teachers = Meeting , Central Off. = Fails||1|
|Excellent at overwhelming teachers.|
Question two asked for an opinion on the recently-completed districtwide Educator Voice meetings held last week between teachers and district leaders. Thirty-two percent of you indicated that you didn’t attend because you were just too busy. Seventeen percent of you felt that it was a waste of time. But, here’s the positive, fifteen percent of you answered, “Don’t think it will change anything, but happy for the opportunity.”
That shows a cracking of the door. Take into account that nine percent of you felt like the district was interested in what you had to say, and I think there is some room here to work. Hopefully district leadership, specifically Director of Schools Dr. Shawn Joseph, heard a concern at one of these meetings that can be used to immediately demonstrate that they were listening. Quick response to a raised concern could possibly begin to turn opinion, or if ignored, cement it. Eyes will be watching.
I would suggest, though, that in the future, extra care be given to arriving on time. As my dad used to say, “Not respecting my time demonstrates that you don’t respect me.”
Here are the write-in votes:
|Just like the parent and community sessions–only for appearances||1|
|Dr. Joseph was late causing me to leave before being able to voice concerns.||1|
|Parents are given more time/grace to attend important events. bad timing.||1|
|Didn’t go, wanted to go, felt it would be frowned upon to be open and honest||1|
|waiting to see if we were heard||1|
|It was too darned late for high school teachers. We are early to bed.||1|
|Didn’t go due to a school function||1|
|Please investigate the Trevecca Doctorate Program||1|
|didn’t go, because I didn’t expect much from them||1|
|Inconvenient time too late after work and nothing would come of it anyway.||1|
|Attended, voiced opinion, but nothing will change|
The last question asked for feedback on the newly-created community superintendent position. Thirty-one percent indicated that you hadn’t seen any change across the district. The number two answer, at 27%, was that they seem to work hard, but you are not sure what they do. I would venture to say those two responses sum up the problem pretty succinctly. The CS’s needed to be a lot more visible in both schools and the community, and their hard work needs to be tied to tangible visible results. Hmmm…. now who else does that apply to?
One person did respond that they were “knocking it out of the park.” I want to thank Damon Cathey for participating in this week’s poll. (That’s just a joke.)
Here are the write-ins:
|They are doing the work Dr. Joseph doesn’t want to appear in person to do.||1|
|Smoke screen while Prince George’s County ruins us|
There you go. MNPS is on fall break this week, so I hope many of you are out recharging your batteries. If you wish to contact me, you can do so at email@example.com. Make sure you check out the Dad Gone Wild Facebook page.
It’s not about test A and test B: Our TVAAS scores are based on the gains of all our students who tested. After the tests have been scanned but before they are scored, each teacher has a list of their students who took the test. To make sure the lists of students “assigned” to a teacher is correct, we have to do “linking” which is a teacher going on to a state site, reviewing the rosters connected to them and essentially saying “yes, those are my students.”
In some cases we indicate if a student was only in our class half the year or if they had excessive absences, in which case they don’t count as much in the TVAAS formula. Anyway, it must be these linking files that got messed up in transit. That’s what the email is saying.
Agreed on TVAAS, though. It’s unfair to teachers because it is so confusing and the formula is secret. It’s unfair to students because what they call “gains” are actually rankings against peers, not actual learning gains. And worst of all, the test never faces public scrutiny- the questions are not released, data on the effectiveness of questions (i.e. what % answer #4 correctly? Were any questions dropped? Do some questions count more than others?) is not released. It could be either the best test ever made or a pile of ____; we’d never know because it’s such a secret.
I agree! These tests are worthless if teachers, students, and parents can’t even see the test questions and the “correct” answers. We never really know how students did because we can’t see the tests or how students answered. So how does this help us as educators? The TVAAS formula is a secret. We teachers have no idea how exactly we are evaluated, so we have no idea how to improve… based upon secret test results.
I have sometimes wondered what would happen if that leadened dirt were closer to important people
When did “vendors” get to determine a teacher’s effectiveness? It just goes to show that testing has become a business/profit-making transaction. Standardized testing no longer has anything to do with education. It is about turning a profit. And as for “new systems,” we get those every year. Just more opportunity for businesses to profit off of public education.
This email to the teacher made no sense and was so filled with gibberish ed. speak. Whoever wrote this email should be ashamed.