cancel
Showing results for 
Search instead for 
Did you mean: 

CFPB released a report comparing FAKOs, FICOs, and VantageScore

tag
cashnocredit
Valued Contributor

CFPB released a report comparing FAKOs, FICOs, and VantageScore

For the geeks amongst us, the CFPB published the first seriouis comparison of FAKOs and FICOs.

 

http://files.consumerfinance.gov/f/201209_Analysis_Differences_Consumer_Credit.pdf

 

There's a lot of good info in this report for those interested in FAKO/FICO correlations as well as FICO to FICO correlations. They studied a really large sample of credit scores.

 

Generally, FAKO's correlation coef was approx .92 with FICOs. Interestingly, VantageScore (v 1.0) correlated somewhat less even though it is a validated credit score with published statistics. This is somewhat surprising since FAKOs are generally "educational scores" not used by lenders and they haven't gone through the processes needed to be accepted by lenders.

 

Perhaps one of the more useful tidbits is that CC industry specific FICOs correlate at .99 to generic FICOs so you can pretty much bet that the generic FICOs availalble here are useful approximations for CC specific ones.

 

One caveat. There was no attempt to determine how well any of the scores actually predicted default risk, their purported goal. One can't tell much about the efficacy of the various scores from the study.

 

Also, the study normalized all scores into a rank ordering percentile so as to compare apples to apples since the scores have different ranges and distributions. Most have a long tail on the high risk side. VantageScore being the exception which is fairly evenly distributed.

 

Another interesting thing is that the correlation is higher for low risk scores so FAKOs, to the extent they can be used as rough proxies for FICOs, are better at the higher score levels than they are for folks with impaired credit.

 

Anecdotally, I've had score variations between FICO and Score Plus, which was studied, that was about 4 standard deviations outside of their published distributions, which reminds me that YMMV too!

 


I have reestablished credit over the last couple years
so my moniker is, well, rather out of date.

WM Discover $1800, WF Plat 12k, Chase Freedom Siggy18k, Amex Plat (60k H/B), Citi AA EWMC 25k
Message 1 of 13
12 REPLIES 12
Revelate
Moderator Emeritus

Re: CFPB released a report comparing FAKOs, FICOs, and VantageScore

Interesting that FICO listed the inquiry penalty as being 5 points for individually counted inquiries (vis a vis non-mortgage / SL / auto within the 30 day window on the FICO '04 model).  I wonder if that's the same for all versions, namely bankcard as compared to classic/standard?

 




        
Message 2 of 13
pipeguy
Senior Contributor

Re: CFPB released a report comparing FAKOs, FICOs, and VantageScore

Thanks for the link - VERY interesting study!

 

I like this quote which shows how FICO might be the "standard" but even using FICO, there is no set standard:

 

FICO’s most current model is FICO 08, but commercial users still use earlier versions of FICO products. Additionally, FICO’s generic scoring models – the most common FICO scores that are developed to predict performance on all types of credit - vary across the nationwide CRAs because the FICO scoring models are designed specifically for each CRA and reflect differences in how they organize and present credit report data.

 

FICO offers industry-specific models for credit cards, mortgages, auto loans, and telecommunication services. FICO models typically generate credit scores in the range between 300 and 850. FICO also builds custom models that are designed for specific companies’ credit underwriting needs.

Message 3 of 13
cashnocredit
Valued Contributor

Re: CFPB released a report comparing FAKOs, FICOs, and VantageScore

I think they meant the 5 points as an average but I'm pretty sure it has to be similar on bankcards v generic. They did state that bankcard specific FICO scores correlated at .99 with generic FICOs. So if they differ it cannot be very much.


I have reestablished credit over the last couple years
so my moniker is, well, rather out of date.

WM Discover $1800, WF Plat 12k, Chase Freedom Siggy18k, Amex Plat (60k H/B), Citi AA EWMC 25k
Message 4 of 13
Revelate
Moderator Emeritus

Re: CFPB released a report comparing FAKOs, FICOs, and VantageScore


@cashnocredit wrote:

I think they meant the 5 points as an average but I'm pretty sure it has to be similar on bankcards v generic. They did state that bankcard specific FICO scores correlated at .99 with generic FICOs. So if they differ it cannot be very much.


You're probably correct on both counts, though to state the correlation factor is 99%, it would've been handy to give us some details as to the average report data as well as we know that not all credit reports are remotely close to equal even if they have the same score under some given version of the model.

 

I didn't look through the report closely enough, but on their initial data set they didn't suggest whom the bankcard-enhanced score was provided from.  Also while mortgage is intrinsically tied to FICO '04, which was likely the overarching theme of the study (their breakpoints had mortgage industry written all over it), I do wonder if the FICO '08 model which has the second (I think, at least subsequent) revision of the bankcard enhanced score, might be a larger disparity.

 

 

 




        
Message 5 of 13
cashnocredit
Valued Contributor

Re: CFPB released a report comparing FAKOs, FICOs, and VantageScore


@Revelate wrote:

@cashnocredit wrote:

I think they meant the 5 points as an average but I'm pretty sure it has to be similar on bankcards v generic. They did state that bankcard specific FICO scores correlated at .99 with generic FICOs. So if they differ it cannot be very much.


You're probably correct on both counts, though to state the correlation factor is 99%, it would've been handy to give us some details as to the average report data as well as we know that not all credit reports are remotely close to equal even if they have the same score under some given version of the model.

 

I didn't look through the report closely enough, but on their initial data set they didn't suggest whom the bankcard-enhanced score was provided from.  Also while mortgage is intrinsically tied to FICO '04, which was likely the overarching theme of the study (their breakpoints had mortgage industry written all over it), I do wonder if the FICO '08 model which has the second (I think, at least subsequent) revision of the bankcard enhanced score, might be a larger disparity.

 

 

 


Yes, there are interesting questions one could have of the more recent vintage scores. The housing crisis changed the time honored pattern of paying mortgages before anything else. That FICO has strategic default risk scores that predict those mortgages at high risk of default while the consumer continues to pay their CC suggests a recent divergence between generic FICO scores and CC specific ones assuming the models have caught up.


I have reestablished credit over the last couple years
so my moniker is, well, rather out of date.

WM Discover $1800, WF Plat 12k, Chase Freedom Siggy18k, Amex Plat (60k H/B), Citi AA EWMC 25k
Message 6 of 13
GregB
Valued Contributor

Re: CFPB released a report comparing FAKOs, FICOs, and VantageScore

I find it difficult to believe that Classic FICO and Industry Specific FICO correlate at .99.

 

All the Industry Specific FICO run the Classic FICO Scorecards and then add two additional Scorecards can affect that score by 50 points in either direction. The stated goal of each industry is to give a score that is equivalent risk as the same Classic FICO but limited to that industry. In other words, a CC Industry FICO of 700 is the same risk for a CC as another person that had a Classic FICO of 700.

Message 7 of 13
cashnocredit
Valued Contributor

Re: CFPB released a report comparing FAKOs, FICOs, and VantageScore


@GregB wrote:

I find it difficult to believe that Classic FICO and Industry Specific FICO correlate at .99.

 

All the Industry Specific FICO run the Classic FICO Scorecards and then add two additional Scorecards can affect that score by 50 points in either direction. The stated goal of each industry is to give a score that is equivalent risk as the same Classic FICO but limited to that industry. In other words, a CC Industry FICO of 700 is the same risk for a CC as another person that had a Classic FICO of 700.


From the report:

"CFPB analyzed credit scores from 200,000 credit files from each of the three major nationwide CRAs: TransUnion, Equifax, and Experian."

 

"FICO Auto scores had a correlation of 0.99, 0.95, and 0.98 for bureaus 1, 2 and 3 respectively with a generic FICO score. FICO Bank Card scores had a correlation of 0.99, 0.99, and 0.99 for bureaus 1, 2, and 3 respectively with generic FICO scores."

 

 

This is by far the largest sample setI've seen so I'm inclined to believe it.


I have reestablished credit over the last couple years
so my moniker is, well, rather out of date.

WM Discover $1800, WF Plat 12k, Chase Freedom Siggy18k, Amex Plat (60k H/B), Citi AA EWMC 25k
Message 8 of 13
Revelate
Moderator Emeritus

Re: CFPB released a report comparing FAKOs, FICOs, and VantageScore

I don't know; the auto-enhanced option being 0.99, 0.95, 0.99 respectively just appears to be laughably off unless out of 200k consumers, 99% of them had an auto-loan on their record.  I can't imagine that's the case.  The "first-time-buyer" penalty is so well known, I just find it shockingly remote that the correlation is that high.  

 

Either everyone is using FICO '08 versions for auto and possibly bankcard enhanced industry specific options that the FICO '04 ones in the study are irrelevant, or something just doesn't seem right with their analysis on that honestly.  Granted anecdotal evidence isn't proof of anything, but at least on the auto forums it comes up over and over and over again to the point it's simply accepted as near absolute truth.  In credit card approvals we also see a wide variation from the best FICO '04 baseline we have available, namely myFICO's EQ Beaction 5.0 (also provided by DCU), and the longer term members who have access to their EX V2 score from PSECU, also seems to disagree with their results as GregB suggests.

 

 

 

 




        
Message 9 of 13
cashnocredit
Valued Contributor

Re: CFPB released a report comparing FAKOs, FICOs, and VantageScore


@Revelate wrote:

I don't know; the auto-enhanced option being 0.99, 0.95, 0.99 respectively just appears to be laughably off unless out of 200k consumers, 99% of them had an auto-loan on their record.  I can't imagine that's the case.  The "first-time-buyer" penalty is so well known, I just find it shockingly remote that the correlation is that high.  

 

Either everyone is using FICO '08 versions for auto and possibly bankcard enhanced industry specific options that the FICO '04 ones in the study are irrelevant, or something just doesn't seem right with their analysis on that honestly.  Granted anecdotal evidence isn't proof of anything, but at least on the auto forums it comes up over and over and over again to the point it's simply accepted as near absolute truth.  In credit card approvals we also see a wide variation from the best FICO '04 baseline we have available, namely myFICO's EQ Beaction 5.0 (also provided by DCU), and the longer term members who have access to their EX V2 score from PSECU, also seems to disagree with their results as GregB suggests.

 


To start with, only one of the three CRAs correlate at .99,   The other two are .98 as well as .95. 

 

But yes, overall it seems surprisingly high. In my own case the scores I had just a year ago were way outside that expected from the published correlations.I had a Plus score that was well above the 90 percentile while EQ FICO was at 40 percentile. This is an extreme rarity according to the report's data.

 

There's another factor at work here. .99 or .98 let alone .95 correlation does not mean the scores are all that close. It allows for a significant number to be quite different from the decile they are compared to. If you look at FAKO/FICO comparisons that correlate at .93 there is more than a 50% chance the scores will be in different deciles and that can map to material FICO score deltas. Especially at the lower end. .99 could still be correct if the differences are skewed towards a small group, the very young just starting out in both credit and first auto.  That combo of short CC history and no auto loan would inidcate beginning driver and that's where the risk differential is.

 


I have reestablished credit over the last couple years
so my moniker is, well, rather out of date.

WM Discover $1800, WF Plat 12k, Chase Freedom Siggy18k, Amex Plat (60k H/B), Citi AA EWMC 25k
Message 10 of 13
Advertiser Disclosure: The offers that appear on this site are from third party advertisers from whom FICO receives compensation.