Comment Detail

Comment

Question A1.1 (1st question): Is there a need to provide new valuation solutions that address industry identified issues of appraiser capacity, turn-times, training, and rural and high-volume market coverage?

The term new valuation solutions to describe the use of unlicensed and/or unregulated alternative work forces is a misnomer. Remember, we had such work forces thirty years ago before FIRREA and the creation of appraisal licensing and regulation.

While it is de rigueur to describe big data, artificial intelligence and machine learning as life changing, real estate big data and machine learning over the last decade have not demonstrated significant improvements to suggest that new valuation solutions driven by algorithms will be safe and sound anytime soon for lenders, investors, and most importantly, homeowners.

A core American value is home ownership. Transparent and auditable collateral due diligence processes and valuations improves confidence in the financial system, promotes expansion of ownership and enhances financial literacy.

Contrastingly, recent academic papers have demonstrated great uncertainty in the current generation of automated valuation models (AVMs) being used. See Principles of Calculating AVM Performance Metrics by Hans Isakson, Mark Ecker and Lee Kennedy (Dec. 2019). These authors upon gaining access to many of the current generation of leading AVMs in the market, concluded,

“Alarmedly the majority of AVMs are underreporting their Forecast Standard Deviations (FSDs). These findings make AVMs appear substantially more reliable than they actually are.”

See also a recent paper entitled, Uncertainty in Automated Valuation Models by Andy Krause, Andrew Martin, and Matthew Fix (Zillow Group). These researchers conclude after exhaustive research on AVMs,

“Existing AVM uncertainty remains highly unstandardized in both terminology and method. In short, there is a lack of specific instructions for AVM producers on creating, measuring and reporting AVM uncertainty.”

In laymen’s terms, these papers reveal AVMs at best as imprecise tools with little to no ability for most users to effectively interpret. Unfortunately, real estate transaction data upon which math and machine are deployed suffers from many foundational challenges.

These foundational challenges result in great uncertainty when attempting to rely upon an AVM. Challenges to using historical real estate transaction data to forecast a single transaction house price include:

• Real estate transaction price data is skewed and fat-tailed making normal distribution metrics unreliable.

• The current generation of AVMs are predominantly driven by geographically and temporally relevant data. One size fits all models have not yet proven effective. While sub-market models perform better in most tests, sub-markets with sufficient transaction volume limit the potential breadth of use of AVMs in the current environment. While machine learning holds promise for the future, significant improvements have yet to be demonstrated.

• The challenges with the importance of temporally relevant data to the current generation of models is that it requires constant updating of the algorithms. Constant updating of the algorithms requires constant monitoring and testing. Currently there is no standardized framework for measuring, monitoring, and testing.

• The de facto metric of Forecast Standard Deviation (FSD) used throughout the industry today is widely known to be a poor metric. Better metrics to measure uncertainty and reliability have yet to be found.

Accordingly, new safe and sound algorithmic valuation solutions do not appear imminent from the research, data and other foundational facts described above. Now I would like to address the impetus for modernization or new solutions, stated issues with appraiser capacity, turn-times, training, and rural and high-volume market coverage.

Capacity, coverage, and turn-times are classical supply and demand challenges. Adam Smith and most first year economic students know how to solve such simple imbalance problems. It is curious FHFA seems so bewildered. While adding appraiser supply in the short term is slightly more complicated than implementing surge pricing, in the long-run additional appraiser supply will find its equilibrium.

Current capacity challenges amidst historically low interest rates and record refinance volume should not be expected to continue indefinitely. If refinance volume were to be dramatically curtailed, would we still have any significant capacity, coverage, and turn-time issues? No.

Question A1.1 (3rd question): What are the risks of these policies and the challenges in implementing them?

Any modernization or new valuation solution or policy that emphasizes cheap and fast driven by uncertain, unstandardized, and unregulated AVMs over quality professional appraisals will risk dramatically over-collateralizing the housing stock and subjecting millions of homeowners to devasting foreclosures when the housing market currently inflated by easy debt goes bust. This time we may not be able to put Humpty Dumpty back together again. The value of things matters.

Industry efforts are underway to develop standardized processes and metrics. Many stakeholders have recently formed working groups to attempt to address these foundational challenges. Until standardized testing and reporting processes are documented by stakeholders and approved by regulators, the use of appraisal waivers and new valuation solutions should be curtailed.

The recent run up in mortgage rates should remind us that the exuberance seen in housing prices has been fueled by low interest rates. Housing price inflation cannot continue in the face of significant increases in interest rates. Interest rates are so low now, small changes upward in interest rates have dramatic impacts on debt service costs and buyer purchasing power. This is not the time to lower the quality of the collateral decisions.