FairTest distorts merit aid data

In the post about peaking in high school, I claimed many people involved in admission don’t distinguish between what is true and what they want to be true. Or maybe they can’t.

Here’s an example.

FairTest is a nonprofit that advocates for the end of standardized tests. In June, it published a report claiming that test scores aren’t as important for merit aid as people think they are.

“Despite the movement to test-optional and test-free policies, there is a lingering public perception that most ‘merit’ scholarships require the use of ACT or SAT scores to determine eligibility. This report seeks to dispel that overly broad generalization while recognizing that the continued use of test results as a determining factor for financial aid represents a major deterrent to college affordability.”

Let’s look at a few flaws in this paper.

Only public colleges

The FairTest report draws the conclusion that test scores don’t matter for merit aid by examining data only from public colleges. In 2021, the most recent year for which the Department of Education publishes numbers, there were 1,892 public colleges and 1,754 private nonprofit ones. I’m ignoring private for-profit colleges because some of them are scams.

FairTest ignores 48 percent of non-scammy colleges in the country in drawing its conclusion. Half isn’t a few.

How many is a few?

Numbers are often the closest approximation of truth, so I like to visit the appendices of papers.

Here are some numbers from this report’s appendix. 32 percent of merit aid programs at flagship state college campuses require SAT or ACT scores. 46 percent of merit aid programs funded directly by state governments require SAT or ACT scores.

A merit aid-seeking student who doesn’t take a standardized test is leaving a lot of money on the table. A third of flagship colleges and half of state scholarship programs wouldn’t be interested in him.

Who gets the merit aid?

Here I’m doing that annoying thing of criticizing an argument but not getting the numbers to refute it properly. Sorry! I think the point is important enough.

The FairTest report only considers which merit aid programs require test scores. The more important question is which applicants get the money. I have a hunch that test-optional merit aid might be like test-optional admission: kind of a smokescreen.

(I’m working on substantiating that hunch, by the way. I’m wrangling with PyPDF2 and BeautifulSoup to yank merit aid numbers out of colleges’ reports, but I’m not very good at it. And ChatGPT code is full of syntax errors!)

Merit aid is self-interested

Public and private colleges give out merit aid because they’re trying to climb the US News and World Report rankings. (Yep, public colleges too.) They try to entice kids with high test scores and grades to enroll, so they can show those good stats to US News.

Rational colleges will try to get the most bang for their buck: the most rankings boost per dollar of merit aid. If we look at how US News calculates its rankings, it looks like spending merit aid on test scores, rather than grades, is a better approach. Test scores directly determine 5 percent of the US News score, compared to class rank (which reflects GPA) with 2 percent.

What does this mean for your kid?

He or she should take a standardized test. Even if your family doesn’t want merit aid or doesn’t want to apply to colleges that give it, the admission odds are better for score-submitting applicants.

For an evidence-based approach to test prep and test strategy, take my online class.

A FairTest employee might now say, “This lady isn’t a disinterested party. She’s only arguing for the importance of test scores because she’s trying to sell you an online course. Talk about scammy!”

My response to this imaginary criticism is one step upstream and relies on the bedrock truth that scarcity is real.

Colleges give out merit aid because they’re all chasing the same smart and conscientious kids. They’re chasing those kids because they want higher US News rankings, which are zero-sum. Colleges want higher US News rankings because they want to make more money. The highest ranked colleges are optimizing for prestige rather than money, but they’re still hustling for those US News rankings.

I observe that admission reflects scarcity and competition, like any human endeavor does. That’s why I suggest your child take a standardized test and why I offer a course about them.

Is/ought confusion is not harmless

Now our imaginary FairTest friend says, “Your logic sounds right, but it makes me feel bad. Can we please imagine that admission is immune from scarcity and competition, because college was fun? Also, test scores reveal tragic socioeconomic problems, so it’s more comfortable to get rid of those tests. There can’t be any carbon monoxide in this coal mine if we kill the canary, right?”

This willful, feel-good blindness harms students and their families. A kid who believes FairTest and doesn’t take a standardized test decreases the odds that she gets into her preferred colleges and that she gets scholarship money.

If you’d like to strategize together about standardized tests or any other element of the application, I’d be delighted to work with your family. You can book a one-on-one session here.