Another One Bites the Dust: MVP Success in Modesto, CA Debunked - "Real World Problem Solving" Sub-score Hardest Hit

"MVP will teach students to solve real world problems!  MVP is equity-based!" they monotonously repeat and tweet and repeat and tweet.  So many times that some actually believe it.

Summary

In this 2017 video, at the 1min 30sec mark, you will hear Ginger Johnson, Associate Superintendent Educational Services, talk about the gains experienced by the new integrated math program in the Modesto District of Stanislaus County, California, where MVP was adopted beginning in 2014 (3 years ahead of WCPSS):

"Educational Services is pleased to report we have seen steady gains in growing student success in Math.  This is evident in the increasing number of students taking advanced 4th year courses as well as students' performance on SBAC math assessment....  we have seen a small steady increase in math proficiency."

Oh, have you?

Background

One uniqueness of this school district is that over 50% of its students who take this assessment are Hispanic / Latino.  So if the MVP claims are true, we should see clear improvements in overall performance and this large subgroup.

California public school test assessment results are available from the CAASPP website.  They administer a state level math assessment in 11th grade.  Scores fall into 4 buckets: Standard Exceeded, Standard Met, Standard Nearly Met, and Standard Not Met.

In the charts below, I am showing results for California, Stanislaus County, and Modesto District, and then comparisons between the district vs. state and district vs. county. As the numbers show, there are gradual downturns in both the raw numbers and the comparison numbers.

District vs County & State - ALL

I like to compare data to a control group, such as a larger population in similar circumstances.  In California, my understanding is that about 2/3 of the state has adopted integrated math.  This is not necessarily MVP, but may explain the declines in state scores.

Sub-scores Not Good

So this is where it gets interesting.  The California test also captures, 3 math sub-scores which are evaluated on a below, near, above standard basis:

1. CONCEPTS & PROCEDURES: Applying mathematical concepts and procedures
2. PROBLEM SOLVING & MODELING/DATA ANALYSIS: Using appropriate tools and strategies to solve real world and mathematical problems
3. COMMUNICATING REASONING: Demonstrating ability to support mathematical conclusions

Now, if you have paid attention to MVP promotional materials, you will know that the big thing they brag about is "critical thinking" and "deeper learning" and the ability to "solve real world problems".   So, to me, that sounds like #2.

Let's look at the data:

Here's #1, CONCEPTS & PROCEDURES: Applying mathematical concepts and procedures

Here's #2, PROBLEM SOLVING & MODELING/DATA ANALYSIS: Using appropriate tools and strategies to solve real world and mathematical problems.

Here's #3, COMMUNICATING REASONING: Demonstrating ability to support mathematical conclusions.

So the silver lining in this - if you're an MVP zealot - is that in sub-score 3 "Communicating Reasoning", there was only a 1% increase in the number of students BELOW STANDARD!  Maybe all that group collaboration is paying off.  Hooray!!   

Meanwhile, the bread & butter of MVP - "deeper learning" and solving "real world problems" - saw an increase in BELOW STANDARD from 39% to nearly 51%!  (That's 1 more kid out of 10 who can't solve real world problems.  Darn!).  

Lagging behind, "Concepts and Procedures" continued its BELOW STANDARD climb from 56% to 62%.   

Hispanic / Latino Subgroup Has Similar Woes

District vs County & State - Hispanic / Latino Subgroup

Sub-scores for Hispanic / Latino Subgroup

So that's it for now.  Similar results as for the ALL data.

By the way, you can jump ahead to the 37min mark to hear from several teachers who speak about MVP.  There are a few fans, and that is expected because some get into it.  But the majority are against.  It's not for everyone as I explained in Myers-Briggs and Why MVP Doesn't Work at Scale.