How accurate is the simulator on this site?

[Patient957]

---

@FICOdawg wrote:

Woukd be interesting to know if myfico simulator also knows what scorecard you're on.

---

Of course it does.  Again, it's their model.  They know how it works.  Why would they have access to the model but decide to publish a garbage simulator that's based on something different?

I have personally verified that the MF simulator accurately simulates scorecard reassignment for new revolver vs no new revolver (i.e. reassignment at AoYRA = 12 months).

[Thomas_Thumb]

I tried the simulator 5 years ago. It was not particularly accurate on an absolute basis. Part of the issue was input restrictions.  Do I think the simulator considers scorecards - not specifically other than clean vs dirty.

Use the simulator to see how changing an attribute of interest might shift score on a relative basis. It can be insightful. However, I don't believe the simulator is the full blown Fico 8 scoring algorithm.

[FICOdawg]

---

@Thomas_Thumb wrote:

I tried the simulator 5 years ago. It was not particularly accurate on an absolute basis. Part of the issue was input restrictions.  Do I think the simulator considers scorecards - not specifically other than clean vs dirty.

Use the simulator to see how changing an attribute of interest might shift score on a relative basis. It can be insightful. However, I don't believe the simulator is the full blown Fico 8 scoring algorithm.

---

That's why I think for simple changes like ageing or simply paying down CC balance and no other changes it's probably pretty close.    Multiple things changing at once and adding CC accounts or loans, no chance.

[Patient957]

---

@Thomas_Thumb wrote:

I tried the simulator 5 years ago. It was not particularly accurate on an absolute basis. Part of the issue was input restrictions.  

---

Yes, they allow limited inputs.  They don't allow direct access to every variable in the model, obviously.  But that doesn't speak to the accuracy of the outputs.  

---

@Thomas_Thumb wrote:

However, I don't believe the simulator is the full blown Fico 8 scoring algorithm.

---

Maybe the full blown version, maybe not.  But it's clear from the data points I've posted that at a minimum, the MF simulator is consistent with Fico 8, even correctly considering scorecard reassignment at AoYRA = 12 months.

Many folks in this thread repeating hand-wavy conventional wisdom about simulators but not offering even a single contrary data point in support said conventional wisdom.

ETA: 

I would be very interested to see some additional DPs.  In particular, I would love to see someone test the supposed scorecard reassignment and loss of ~20 points on young clean profiles when AoOA = 36 months.

I know we're mostly curmudgeons here myself included, but if anyone has a clean young profile for testing this, it would be interesting.  Need a clean profile with AoOA < 36 months and have access to the MF simulator (which only comes with a paid product, if I'm not mistaken.)

[JoeRockhead]

---

@Patient957 wrote:

---

@Thomas_Thumb wrote:

I tried the simulator 5 years ago. It was not particularly accurate on an absolute basis. Part of the issue was input restrictions.  

---

Yes, they allow limited inputs.  They don't allow direct access to every variable in the model, obviously.  But that doesn't speak to the accuracy of the outputs.  

---

@Thomas_Thumb wrote:

However, I don't believe the simulator is the full blown Fico 8 scoring algorithm.

---

Maybe the full blown version, maybe not.  But it's clear from the data points I've posted that at a minimum, the MF simulator is consistent with Fico 8, even correctly considering scorecard reassignment at AoYRA = 12 months.

Many folks in this thread repeating hand-wavy conventional wisdom about simulators but not offering even a single contrary data point in support said conventional wisdom.

ETA: 

I would be very interested to see some additional DPs.  In particular, I would love to see someone test the supposed scorecard reassignment and loss of ~30 points on young clean profiles when AoOA = 36 months.

I know we're mostly curmudgeons here myself included, but if anyone has a clean young profile for testing this, it would be interesting.  Need a clean profile with AoOA < 36 months and have access to the MF simulator (which only comes with a paid product, if I'm not mistaken.)

---

So I just went to the simulator and it tells me that no matter whether I increase my credit card limit by $1,  or by $35k I am taking a hit to my scores of -10 with EQ, -5 with TU & EX...  Since I've never taken a score hit from increasing card limits, my guess would be it's because the system assumes I'm taking a HP. 

Oddly enough, it shows the same identical score losses for taking out any type of new loan.  Mortgage, Student, Auto, or Personal.  No matter what the loan amount is it doesn't influence, or change the simulated score loss. 

All data points showing the use of generalized scenarios that aren't one size fits all. 

[FICOdawg]

---

@Patient957 wrote:

---

@Thomas_Thumb wrote:

I tried the simulator 5 years ago. It was not particularly accurate on an absolute basis. Part of the issue was input restrictions.  

---

Yes, they allow limited inputs.  They don't allow direct access to every variable in the model, obviously.  But that doesn't speak to the accuracy of the outputs.  

---

@Thomas_Thumb wrote:

However, I don't believe the simulator is the full blown Fico 8 scoring algorithm.

---

Maybe the full blown version, maybe not.  But it's clear from the data points I've posted that at a minimum, the MF simulator is consistent with Fico 8, even correctly considering scorecard reassignment at AoYRA = 12 months.

Many folks in this thread repeating hand-wavy conventional wisdom about simulators but not offering even a single contrary data point in support said conventional wisdom.

ETA: 

I would be very interested to see some additional DPs.  In particular, I would love to see someone test the supposed scorecard reassignment and loss of ~20 points on young clean profiles when AoOA = 36 months.

I know we're mostly curmudgeons here myself included, but if anyone has a clean young profile for testing this, it would be interesting.  Need a clean profile with AoOA < 36 months and have access to the MF simulator (which only comes with a paid product, if I'm not mistaken.)

---

I'll be able to test as I'm 1/12 for pulls, albeit I have much older profile.     The only variable that will change is 6 months older for all accounts and my CC utilization will drop.     Right now the simulator says +5 for the aging and +50 for the utilization.   Current AAoA is 8 years 9 month AAoA.     Current score 766 

[FICOdawg]

---

@JoeRockhead wrote:

---

@Patient957 wrote:

---

@Thomas_Thumb wrote:

I tried the simulator 5 years ago. It was not particularly accurate on an absolute basis. Part of the issue was input restrictions.  

---

Yes, they allow limited inputs.  They don't allow direct access to every variable in the model, obviously.  But that doesn't speak to the accuracy of the outputs.  

---

@Thomas_Thumb wrote:

However, I don't believe the simulator is the full blown Fico 8 scoring algorithm.

---

Maybe the full blown version, maybe not.  But it's clear from the data points I've posted that at a minimum, the MF simulator is consistent with Fico 8, even correctly considering scorecard reassignment at AoYRA = 12 months.

Many folks in this thread repeating hand-wavy conventional wisdom about simulators but not offering even a single contrary data point in support said conventional wisdom.

ETA: 

I would be very interested to see some additional DPs.  In particular, I would love to see someone test the supposed scorecard reassignment and loss of ~30 points on young clean profiles when AoOA = 36 months.

I know we're mostly curmudgeons here myself included, but if anyone has a clean young profile for testing this, it would be interesting.  Need a clean profile with AoOA < 36 months and have access to the MF simulator (which only comes with a paid product, if I'm not mistaken.)

---

So I just went to the simulator and it tells me that no matter whether I increase my credit card limit by $1,  or by $35k I am taking a hit to my scores of -10 with EQ, -5 with TU & EX...  Since I've never taken a score hit from increasing card limits, my guess would be it's because the system assumes I'm taking a HP. 

Oddly enough, it shows the same identical score losses for taking out any type of new loan.  Mortgage, Student, Auto, or Personal.  No matter what the loan amount is it doesn't influence, or change the simulated score loss. 

All data points showing the use of generalized scenarios that aren't one size fits all. 

---

Yes simulator would be upgraded if it let you assume CLI was via SP vs HP.   That's a major gap in the sim.