MVP Ground-Zero Math Performance Data Exposed, and it Ain't Pretty: An Analysis of American Fork Junior & Senior High Math Trends

An Illustration to Lighten the Mood

Note: Skip ahead to "Background" if you want to jump to the data

What if I told you I had a car to sell you - NO, GIVE YOU - that will bring you an increased feeling of #equity to all your driving needs!?  In fact, you will gain "deeper driving" experiences which enable 21st century transportation skills like you've never seen before!  No more "rote driving!"  We're talking "real world" "critical driving" improvements.   In fact, this new car satisfies all 8 practices of sound driving as identified by the National Council of Car Drivers - get this - by focusing only on ONE of them: Practice #8 - Use the steering wheel.  You heard me right.  By focusing on using the steering wheel the way that I'm going to teach you, you are going to be driving like never before!  I'm willing to GIVE you this car - TODAY - if you are ready to engage in this really popular type of driving discourse.  Soon enough, you'll be driving on DIS-course AND DAT-course!  Get it?  DIS-course?  There's only one catch.  You have to pay ME to teach YOU how to use that steering wheel MY WAY.  And that is the ONLY way you can use it.  Otherwise, Hey, fuggedabout it.    

You say, "All those benefits sound great, but will it help my gas mileage?  I'm getting 58 MPG now, which is pretty good.  And that is my main priority when it comes to transportation!"  

Is that all you think about?  Measuring results?  You're using traditional driving thinking.  I assure you, speaking as an expert in this new drivagogy, you will be delighted in this new "discovery" method of driving the new way.  (Drivagogy is "the method and practice of driving, especially as an academic subject or theoretical concept.")

Let me tell you, since I have been using my new drivagogy method, my MPG improved from 58 to 57.6 in less than 5 years! 

"But wait.  That's not an improvement.  That is basically flat - actually a slight decline," you say.

You got to understand.  The drive performance improvements don't come right away.  You have to be patient.  After a little productive struggle, you'll get the hang of it.   Here - let me offer you this tasty beverage.  Oh Yeaaaaah.  Here at M.V.P. Motors, you have to taste this to believe it.

So, what would you do?  I'm guessing that most sane people would drive, walk or RUN away from such a proposition.  Unfortunately, that little story is closer to the truth than fiction.

Background

Mathematics Vision Project, LLC, was founded in 2013 by principal Travis Lemon.  In addition to being the thought leader behind MVP, Mr. Lemon currently teaches at American Fork Junior High in American Fork, Utah, where he has taught since 2000.   According to Karen Feld, who is also a math teacher at AFJHS, "Travis is an amazing mathematics educator who is focused on the learning and success of all students."  Additionally. Ms. Feld notes, "My team and I use the MVP materials. These materials are excellent. The MVP materials allow students the opportunity to engage in the 8 Mathematical Practices and allow me the opportunity to include the high leverage and research based Effective Teaching Practices (NCTM, 2014) in my classroom. We have had great success and our students demonstrate high levels of mathematical understanding."

While I don't question Mr. Lemon or Ms. Feld in their passions for being teachers, I have a responsibility to look at the data in light of what we are seeing in Wake County.  My thinking is this: American Fork Junior High School has CLEAR ADVANTAGES OVER EVERY OTHER SCHOOL OR SYSTEM ADOPTING MVP.

1. Expertise: As the creator, author, and #1 user of MVP, Mr. Lemon is arguably THE WORLDWIDE EXPERT on the subject.  
2. Buy-In: Additionally, not only is he one of the MVP teachers at the school, he apparently has an adoring set of peers who wholeheartedly embrace the MVP curriculum.  
3. On Demand Help: Furthermore, if there were ever any questions or issues to work through, Mr. Lemon could provide on-demand professional development in the Teachers' Lounge or cafeteria with his peers.  

Certainly, for these 3 reasons, this is a PERFECT SCENARIO in which to deploy MVP and observe the mathematics comprehension bliss it will deliver to students!  I think that most logical people would expect his school to produce the BEST RESULTS POSSIBLE compared to any other in the world.

Disclaimer

I have great respect for all teachers who give their all each and every day for their students.  I know that no curriculum is perfect and no measurement system is perfect.  Sometimes we may disagree about how and why things should be done in the field of "educating the children."  Any observations or critiques I make below are not intended to be aimed at teachers or students, but to the decisions that perhaps should be reconsidered in hindsight.

Utah's Data Gateway

The Utah State Board of Education provides a data gateway for citizens to examine school results.  You can run a report for a school and you will see the school's results in context of up to 19 other schools that are similar.  This "similarity comparison" is very insightful because it groups schools using a "general coefficient of similarity," taking into account about 11 variables contextualizing school attributes and student demographics.

Utah measures % Proficient (percent of students who are grade proficient in a subject) and Median Growth Percentile (MGP).  There is a video which explains MGP, but I will summarize.  Student Growth Percentile (SGP) measures each individual student's percentile compared to similar performers in the prior year.  MGP, however, measures how much students are improving from year to year as a group.  An MGP of 55 would mean that for the population considered, the median SGP was 55.  This would mean that the median SGP student improved better than 55% of his/her peers at the state level.  In short, MGP measures year to year improvement.  Utah's objective is that an SGP of 40 or higher is considered to have made sufficient growth.

2014/15 to 2017/18 Data

So let's look at the 2014-15 data for American Fork Junior High School.  Here you will see the Proficiency is 58% meaning 58% of students were grade level proficient in math.  The MGP measuring growth from the prior year was 53.  Meaning that when we take all the AFJHS students and sort their SGPs, the median one was 53.

If we fast forward to 2017-18, let's see those results.  You will note that there is not much change in proficiency from 2014-15 to 2017-18.  However, the MGP declined from 53 to 48.  Still OK, but declining.

Pulling in the two years in between, the 4 year trend in proficiency is: 58 - 60 - 56.4 - 57.6
The 4 year trend in MGP is: 53 - 52 - 48 - 48.

Pre 2014 Data

Now, when I saw this, I had some concerns.  I am estimating that AFJHS started using MVP around 2013.  After all, I would have expected Mr. Lemon to have a product nearly in hand before forming the Mathematics Vision Project, LLC.  Anyway, the other thing that happened in the 2012-2013 timeframe is "common core".  The data above doesn't go back before 2014.  However, there is another source which shows some older data.  This particular view seems to only be current through 2015-16.

As seen in the graph below, before Common Core, proficiency was on the rise from ~77% or so in 2010 up to around ~90% in 2013.  Then Common Core and/or MVP became the law of the land at AFJHS.  Most data I've seen for other schools shows a decline in scores once Common Core came into play.  So I will not focus so much on that because this report is not about Common Core.  It's about MVP.  And for all intents and purposes, it seems that the 4 years of data available at the original data gateway aligns to MVP years 2-5 at American Fork Junior High School.

Comparing & Contrast with Two Other Schools

To complete this analysis, I looked at two other schools.  The first was American Fork High School.  Approximately 100% of the students from AFJHS feed into the high school of the same name.  Additionally, about 50% of students from another junior high feed into the same AFHS.  This is illustrated as shown.   My thinking was that the MVP students leaving the junior high could also be measured in high school.  Even though we don't know the curriculum of the "other junior high school" about 2/3 of the math students at American Fork High School are from AFJHS.  So the data would be interesting if nothing else.

For the record, I reached out to teachers at the American Fork High School and asked if they used MVP.  They were very nice and responded promptly.  The answers included these comments:

• We do not use MVP. If we do, we use it sporadically, maybe once or twice each term
• We use our own curriculum and some teachers choose to insert an MVP lesson or two into our curriculum where they see necessary but it is not common
• From my brief interactions with it, we as a department saw many issues with it and haven’t ever pushed for it at our school.
• I personally use MVP very little.  At the high school level, I find tasks are a great tool for discovering some concepts, supplementing others, and some just require old fashioned direct instruction.  
• As a department at our high school, we are working to incorporate more tasks because most of us teachers use primarily direct instruction, and we feel that tasks could deepen understanding.

The second other school I looked at was Fairfield Junior High, which is in Kaysville, Utah, in the Davis County District.  Why Fairfield?  If you will refer back to the 2017-18 comparison graphic for AFJHS, you will see Fairfield at the top of the list.  Again, Fairfield is a school which is similar to AFJHS.  In fact, the website shows them as 91% similar.  I wondered how they fared over the prior 4 years, so I pulled that data too.

Let's see the results.  Let it soak in.

Observations

So here are some observations:

• As noted above, you see that American Fork Junior High is relatively flat on Proficiency, and down on Growth (MGP).  
• But look at the high school (where 2/3 of its students are from AFJHS).  Down...  Down... Down... on both Proficiency and Growth.  Just to be clear, we don't have a perfect correlation that says that MVP students at AFJHS are driving the declines at AFHS, but there seems to be some correlation based on the 2/3 of the high school students represented being from the MVP junior high.  If I was a parent of those AFJHS or AFHS students, I would certainly want to know. 
• At the high school, Proficiency has dropped 17.4 points from 53.8 to 36.4.  That is about a 1/3 drop and reflects devastating news to the students and parents affected.
• One might be tempted to think that if the high school would only adopt MVP, then the MVP kids coming from the junior high school would do OK.  But think about that logic.  The direct teaching style is what students will experience in college.  If they are not making the transition from junior high (discovery) to senior high (direct), then I shudder to think how they will handle math in college.
• Growth at this high school is down 16 points from 55 to 39, another drop of nearly 1/3.  The 39 is BELOW what Utah considers acceptable growth.  Is anyone there looking at these results and wondering, "W. T. H?!!"
• Now, we look at Fairfield.  Their proficiency has steadily increased over this same period from 57.5 to 63.7 which is 6.2 points, or a 10.8% increase.  Growth has dropped a tad from 65 to 60, but still the Growth results are outstanding and well over 50.

I'm not shy about reaching out to strangers, so I did.  I contacted teachers at Fairfield Junior High and wanted to find out what their secret to success was!  After all, isn't this one of the ways education leaders should select new ideas and curricula?  Find out what works!

I met with Lori Kalt there, and here's what she told me:

1. A/B block schedule, so we see our students every-other day for 80-minute class periods.
2. We defied the recommendation and push from our district to use their adopted curriculum (Carnegie) and we developed our own curriculum.  (By the way, if you compare Fairfield among all 26 junior high schools in their county - they are #2 in math proficiency for 2017-18.  Lori noted that the #1 school, Kaysville Jr. High, does not use the block schedule for math - so their math class-time is considerably longer.)
3. We employ lots of best practices in our curriculum which includes lots of direct instruction, skill-based practice for fluency (on skill-based concepts), whole-class activities and dialogue to help students grasp and understand conceptual things, analyzing mistakes, fine-tuning understandings using mini white boards, and daily short quizzes on each standard. 
4. We use standards based grading and students have multiple opportunities to show mastery on concepts - not unlimited opportunities but students can retake quizzes a few times
5. We make sure all students are fluent on foundation skills such as solving equations, constructing equations, graphing equations etc.

She also noted, 

We are currently receiving a LOT of pressure from our district to employ the use of tasks more frequently.   I work with an amazing group of math teachers and we are all like-minded and collaborate a LOT with each other.  We are hesitant to wholly employ the latest fad(s) in education which right now seems to be tasks, discovery, project-based learning, and personalized/online learning.

Conclusion

So there you have it.  Who would have thought that having human teachers create a curriculum from scratch could beat a bill of goods and services which has cost Wake County over $1.5M?

Who would have thought that teaching using a mixture of various crazy ideas such as:

• direct instruction, 
• skill-based practice for fluency (on skill-based concepts), 
• whole-class activities and dialogue to help students grasp and understand conceptual things, 
• analyzing mistakes, 
• fine-tuning understandings using mini white boards, and 
• daily short quizzes on each standard 

would have resulted in not only UP-TRENDING results, but clear leadership compared to similar schools?

Who would have thought that those results would be far superior to ground-zero of the latest and greatest trend in so-called innovative (yet-unproven) constructivist mathematics pedagogy?  

I tell you who: ME... and many other parents and teachers in Wake County and around the US and Canada who know better than to risk our students' futures on a tweet filled with feel-good hashtags, or on a chance meeting at a dog and pony show at a math conference.

You know what WCPSS should do NOW that they know the truth about MVP?

Fuggedabout it.