On Monday, July 8th, we made some updates to our Best Colleges, Highest Paid Grads and Best Colleges for the Money rankings. These changes were aimed at improving the quality of the rankings, and in the case of the Best Colleges for the Money, the practical usefulness of them.
Best Colleges Rankings Update
With this update, a change was made that improves the earnings estimates we use in the overall Best Colleges rankings.
PayScale.com provides us with earnings data by major for graduates from each college or university. For each institutions' majors, we compare the earnings of its grads to the overall average for all graduates of the major. This is the ‘earnings boost’ metric we use in our rankings, which we calculate for both early and mid-career earnings. A positive boost indicates that graduates in that major earn more than the average, while a negative boost suggests graduates earn less.
The previous method for calculating the boost assigned to the school, either negative or positive, was to average across all the majors at a given college or university. While this is a perfectly valid approach, we saw an opportunity to improve upon it with the following changes.
Weighted Average by Student Counts
Rather than take a simple average of the majors, treating each one equally, we decided instead to weight them based on how many students graduate from each school for that major. We think this is a more accurate representation of the average and removes scenarios where a major with a particularly high or low earnings boost, but only a small number of students, can impact the average more than it should.
Estimating Missing Earnings Data
There are times when Payscale does not have sufficient data to indicate average earnings for majors at a particular college or university. The previous rankings excluded these majors from the calculation. Unfortunately, this sometimes ignored a school's important majors.
Rather than completely ignore these majors, we chose instead to come up with a model that estimates all earnings values based on the information we do have. This new model is used to estimate earnings information for all majors at a college or university, when we have a sufficient amount of PayScale data to do so. A number of advanced statistical techniques are employed to accomplish this. These techniques consider a number of things, including the earnings data for other majors at the institution, as well as earnings information for that major across all colleges and universities.
The Net Result
The two changes above result in an updated and more useful set of earnings and earnings boost calculations. This new data was then incorporated into the latest rankings. Some schools have gone up, some down, but in general, the colleges and universities didn’t shift too many places from where they were previously.
One exception to this is Amherst, which went from #19 overall to #1 on the Best Colleges Overall, nationally. This school happened to suffer from the issues noted above, more than others. While a jump of 19 spots looks significant, out of a pool of 1,300+, it is not. The reality is that the scores of all the colleges and universities in the top 20 are very close, and therefore even a small shift in actual numbers can have what looks like a disproportionate impact on the exact ranking.
Rankings and Numbers
Keep this in mind as you look not only at our college rankings, but all the rankings out there. College rankings and lists can give you a rough, relative sense of an institution's quality. However, a difference between a ranking of #10 vs. a ranking of #20 does not indicate that the first college or university is twice as good as the second, no more than a difference between #1,010 and #1,020 would. The reality is that all schools within 20 or so spots are of similar quality. This is certainly true of our top 20, all of which are outstanding schools.
Highest Paid Grads Rankings Update
The change to estimate missing earnings data for schools also affects the Highest Paid Grads rankings created for each major. The net result is better estimates and just as importantly, a larger number of schools included in the rankings. When there are still colleges and universities missing from the ranking for a particular major, it is because there was too little data to reasonably estimate a value.
In addition to the changes noted above, one bug fix was also rolled out. A Highest Paid Grad rankings for each major (i.e. Chemical Engineering) is created, and also a rolled-up version at the field of study level (i.e. Engineering). There were a few fields of study displayed without data, even though the majors below clearly contained data. This display bug was corrected and those fields of study now appropriately show earnings data and rankings, compiled from the majors within the field.
Best Colleges for the Money Rankings Update
The goal of this ranking is to compare the quality of education to its cost and identify colleges and universities where you are likely to receive the best quality education for the least amount of money. The quality of a college or university represents half the calculations used to determine the Best Colleges for the Money. Quality is determined by the Best Colleges rankings, and therefore, the changes noted for those rankings also resulted in a recalculation of this rankings list as well.
In addition, fundamental changes were made to methodology, based on feedback we received from students, parents and higher education counselors. Many found this rankings list hard to understand. In fact, few people outside of College Factual's data science team, including other employees of College Factual, truly understood them!
The feedback centered around a couple points:
- ‘Estimated Value of Degree’ and an ‘Estimated Savings’ numbers were often misinterpreted and therefore, could be misleading
- Colleges and universities with low quality scores overall were ranking highly in value, due to having some of the lowest net prices nationwide. However, for many students and parents, an extremely low quality college does not exemplify a 'good value', regardless of the price. Therefore, the quality of a college or university needed to be factored more strongly into the algorithms.
While the previous methodology was accurate and informative, it ultimately did not meet our standards for usefulness. The company's goals are not to make rankings just for the sake of rankings, but rather, to create useful and customizable tools that help families make better college decisions.
The updated methodology is detailed in this overview. However, some of the changes made are outlined below:
Affordability Rank
The estimated degree cost of the school is ranked based on how affordable it is. The lower the price, the better the affordability rank. We then compare that to the quality of the school. Schools with a great combination of quality rank and affordability rank are awarded the best position. This better highlights schools that offer both high quality, and great affordability, and is much easier to understand.
A lot goes into calculating both of these metrics, but a couple interesting nuances are worth mentioning.
- Schools that inflate their ‘sticker price’ and then award students significant discounts (aid) to reduce attendance cost are not rewarded more than those institutions that simply set affordable pricing to begin with.
- As ranking well requires both quality and affordability, low quality schools cannot rank too highly when it comes to value. At best, an extremely cheap institution may rank as a good to fair value, but will not rank as a "best value".
- On the flip side, high quality colleges and universities that charge a similarly high price are going to rank as a fair value, which is what they are (you are getting what you pay for). However, these schools are not a bargain, and so they will not rank as a best value.
Future Rankings and Tools
At College Factual, our goal is to create customizable rankings and personalized tools that will help families select the best college or university for their needs. We continuously strive to improve upon what we've built, and never shy away from revising what has already been created. If we discover a better way to help a student and her family make one of the most important decisions of her life, we are going to put that tool in students and parents hands as soon as possible, rather than wait an entire year for another 'rankings season'.
Update Notifications
To receive notifications when rankings change, or when new tools are announced, subscribe to our newsletter. If you are a college administrator and would like advance notice of rankings changes, please sign-up via the following page: http://www.collegefactual.com/about/marketing-opportunities/