How to Read NCARB Fail Reports

You completed an ARE® 5.0 division and failed it — you now puzzle over the fail report, reliving the trauma and compounding the drama. NCARB authors its fail reports to be neither transparent nor intuitive to read. I find them to be, well, um … unintentionally misleading.

First, to pass the exam, you’ll need to answer about two-thirds of the test items correctly, regardless of what section the questions come from. There is no “failing a section” or “passing a section” of a division. 

To illustrate with an example, let’s imagine that you need to reach one million dollars in retirement savings to retire (to “pass” a retirement exam). You own a retirement account with some stocks and some bonds. It doesn’t matter if most of your gains come from the basket of bonds you own or if most of your gains come from the stocks you own. 

As long as you have the minimum from both sources combined, you win. Your fail report may show you lacking in stock performance, but that can be easily overcome by strong bond performance. If you “failed” to save enough money for retirement, you can try again and wait a few more months for your account to swell, but it won’t matter if your gains over those few months come from the stock basket or the bond basket. . . as long as they come.

But it’s more complex than that because some ARE 5.0 exam division sections are worth far more than other sections. PPD has 100 test items. Of those, Section 4, “Project Integration of Programs and Systems,” may represent fully 38 of those 100 test items you’ll find in PPD; by contrast, Section 5, “Project Costs and Budgeting,” may represent only eight of those 100 test items.

In your retake, is it better to answer a higher proportion of questions from Section 4 or a higher proportion of the questions from Section 5? The answer, of course, is “it’s almost certainly going to help you more to improve on your performance in Section 4 than it will to improve your performance in Section 5, regardless of your performance last time in those two sections.” Again, with the retirement savings analogy: If 80% of your retirement account is in stocks, and 20% is in bonds, which do you spend most of your time researching, the stock market or the bond market? If more of the next exam’s questions will come from Section 4, “Project Integration of Programs and Systems,”  than from Section 5 “Project Costs and Budgeting,” you should spend more time studying Section 4.

How To Study For a Broad Subject

Now that we’ve established that you’d prefer to improve in Section 4, “Project Integration of Programs and Systems,” how exactly does one study for a subject that broad? That might include soils, HVAC or roofing. So even once you’ve hacked the fungible nature of the test items and the weighted nature of the ARE 5.0 exam division sections, your fail report still doesn’t really tell you what you missed last time and need to study more of before tackling a retake. Finally, in an effort to be more transparent, the NCARB began sharing a column in the fail reports in 2022 called “Passing Candidates % Correct” for each section. They meant well, but many exam-failers wrongly believe that next time, they need to get that column’s percentage correct in each content area row in order to pass themselves. This is not true because: 

1. Each question counts for one point in the whole exam, as described above, so it doesn’t matter which content area the extra points come from. 

2. This column doesn’t relay the “cut score” that you need to pass, but rather the average score of those who exceeded the cut score

Do a thought experiment: two people passed an NCARB exam with a cut score–the percentage right required to pass–of 66%. One of the two earned 67% and passed and the other earned 100% and passed. What is the “Passing Candidates % Correct?” That number will always be higher than the “Percent you need to get correct to pass."

So every question has the same value, even though you wouldn’t think so by reading the fail report. Doing poorly on one particular section may not mean that topic is the best to focus on for the retake, even though you wouldn’t think so by reading the fail report. NCARB names the sections in a way that makes them particularly unhelpful for choosing topics to study next time.

What looks like the percentage of questions correct you’ll need to pass is actually much higher than the cut score. Then how should you study for a retake? The same way you studied for that ARE 5.0 division last time . . . efficiently going through the material and without paying much attention to your last fail report.