Glass Box not Black Box

Glass Box not Black Box

Vincent Costa

Vincent Costa, CFA

Head of Global Quantitative Equities

Gareth Shepherd

Gareth Shepherd, PhD, CFA

Co-Head Equity Machine Intelligence, Portfolio Manager

Demystifying the role of machine intelligence in stock selection.

In our previous blog “Voya Equity Machine Intelligence: Human Plus Machine”, we described how machine learning can be applied to the task of finding persistent patterns in company fundamental data. We also emphasized how machines alone are not enough; a combination of “human + machine + better process” is needed.

Here, we will explore the human element of our process (known as “Human in the Loop”), and how this helps lead to strong model interpretability and a reduction in model failures.

Indeed, the deliberate, albeit selective human input throughout the process helps to shed light on what otherwise can end up being an inscrutable “black box”. We call this human-augmented AI, and it leads to a much more transparent process – or a “glass box”.

The road to interpretable models: Human in the Loop

While machine learning is at the core of our stock selection process, the approach is by no means machine only, as shown in the diagram below. Our virtual analysts and virtual traders are solely responsible for picking stocks; the human team oversees all processes and controls both the inputs (data) and outputs (risk managed portfolio).

Figure 1. Our AI stock selection process begins and ends with human decision makers

Before the machine can deploy its algorithms, it is necessary for the human team to identify and scrub complex data sets consisting of 10,000 unique data points per company (spanning across macro and sentiment, fundamental and ESG data). While perhaps this is the least sexy part of the process, data preprocessing is vital for success. Once complete, the next step is to move from data to features, involving a hybrid human + machine effort. The resulting feature set (think complex financial ratios) help the virtual analysts find predictive patterns based on a deep, holistic view of each company in the universe.

This process, dubbed feature engineering in the jargon, means that the input variables (252 in total) are explicit, rules-based, and fully transparent. Moreover, they are carefully curated – by humans – to ensure that each feature is backed by an economic rationale and is not conflating noise with signal.

We also have the added benefit of working with Voya’s team of fundamental and quantitative analysts, who improve upon our insights and feature construction.

To close out the process, we again rely on our human portfolio managers to oversee risk management and trade execution. Every trade is reviewed by the team and they will override a trade if deemed necessary, such as if there is a corporate action which is not yet captured by the model. This step is key to managing the risk of an algorithmic error, which is sometimes experienced by high frequency traders (and a popular hobgoblin of those critical of machine-only processes!).

The Final Product: A Glass Box Approach

EMI’s proprietary technology allows human portfolio managers to interrogate the system to determine the explicit rationale of each buy and sell decision. Indeed, they can go into the same level of detail that a fundamental analyst would in support of a bottom up stock-picking strategy – minus the human bias. A caveat – this does not mean that AI can replace human analysts (who, for instance, may be better placed to forecast long-term trends or structural changes) – but it does mean that AI can be geared to make fewer mistakes than most humans.

By way of background, explainable AI (XAI) is the class of systems that provide visibility into how an AI system makes decisions and predictions. XAI can explain the rationale for the decision-making process, shows the strengths and weaknesses of the process, and provides a sense of how the system will behave in the future.1

For instance, a machine learning model can be either directly interpretable or approximated with a decision tree. The IF-THEN logic of the decision tree can provide an explanation of the relative importance of features influencing the model’s predictions. Diagnostic techniques can then be employed by the EMI team to:

1. Generate insights on the importance of specific features

2. Glean a traceable path leading to the model’s predictions and/or

3. Gain a granular view on the impact of a particular feature both on individual holdings and across industries

Such explanations are helpful for client communications and enhance our ability to continually improve the process. For example, these insights allow the EMI team to enhance the model through targeted feature engineering, modification of the underlying architecture, and/or revision of data resources.


On paper, this may sound quite complex, and it is perhaps best demonstrated through an example. In Figure 2, we illustrate this via analysis of a recent purchase of Wells Fargo & Co (ticker: WFC) – a position which was initiated in October 2020 as a top-five portfolio holding.

Figure 2. AI sees a potentially brighter future for WFC than most human analysts

Source: FactSet. Wells Fargo has outperformed the S&P 500 by 13.25% since October 5th (ending January 31). ** Weight changes greater than +/-15bps excluded

From October 5 2020, several of EMI’s virtual analysts found the stock to be attractive because it fell into a broad predictive pattern: a cheap, distressed, fallen angel where sentiment is overly negative, yet fundamentals appear to be slowly turning around and the stock price may be bottoming (i.e. a classic “Turnaround” play).

How do we know this? By using inhouse XAI tools to make the decision-making logic of the virtual analyst explicit, including features which underpin the pattern. For illustrative purposes, these features include:

  • Value: Cheap relative to peers and history according to various adjusted metrics such as EV to free cash-flow, price to earnings and price to book
  • Quality: Dozens of important balance sheet and cash-flow statement items are improving, as are various proprietary ESG metrics such as governance, compensation and other issues
  • Sentiment: Historically low ratings and price targets from sell-side analysts; various market signals show signs of investor capitulation
  • Technicals: Price bottoming, relative strength, and long-term negative momentum (reversal signal)

In addition to the basic investment rationale described, we can clearly identify why a stock was bought or sold at a particular time from the virtual traders. In the case of Wells Fargo, sentiment is so negative that human nature could deter even a skilled analyst from purchasing the stock despite improving fundamentals. Conversely, our virtual analysts and traders remove emotion from investing and do not shy away from opportunistically buying into a story identified as a potential turnaround.

Finally, and perhaps more importantly, the technology allows for performance attribution at the level of the patterns themselves – to determine what is really driving portfolio returns. Coming back to WFC, the original purchase represents an opportunistic turnaround play with a time horizon of 3 – 24 months. In this case, within 4 months of initiating the position, WFC performance is 22.76% versus the S&P 500 return of 9.51% (through end of January 2021).

Figure 3. A snapshot of some of the patterns cited by AI analysts (NB this is but a small sample of the clusters of predictive patterns available)

“Invest in what you know… and nothing more.” ~ Warren Buffett

While traditionally there has been a wide chasm between quantitative and fundamental investing, we believe the next frontier lies in building a strong bridge between machine and human. This goes beyond a ‘quantamental’ approach which blends both, to a process that leverages the best of both worlds. More specifically, our aim is to combine the breadth and dynamism of quantitative equity investing with the depth and analytical rigor normally seen in top human managers.

Further, one of the keys to success for any investment strategy is the ability to not only identify the drivers of performance, but also to communicate these drivers to clients. The transparency in the EMI approach does just that: it allows for the decision-making rationale – i.e. the “patterns” that drive alpha generation –to be explained, putting to rest the critique that machine learning has to be an impenetrable “black box”.

IM1522142

The S&P 500 Index is an unmanaged capitalization-weighted index of 500 stocks designed to measure performance of the broad domestic economy through changes in the aggregate market value of 500 stocks representing all major industries.

This commentary has been prepared by Voya Investment Management for informational purposes. Nothing contained herein should be construed as (i) an offer to sell or solicitation of an offer to buy any security or (ii) a recommendation as to the advisability of investing in, purchasing or selling any security. Any opinions expressed herein reflect our judgment and are subject to change. Certain of the statements contained herein are statements of future expectations and other forward-looking statements that are based on management’s current views and assumptions and involve known and unknown risks and uncertainties that could cause actual results, performance or events to differ materially from those expressed or implied in such statements. Actual results, performance or events may differ materially from those in such statements due to, without limitation, (1) general economic conditions, (2) performance of financial markets, (3) changes in laws and regulations and (4) changes in the policies of governments and/or regulatory authorities. The opinions, views and information expressed in this commentary regarding holdings are subject to change without notice. The information provided regarding holdings is not a recommendation to buy or sell any security. Fund holdings are fluid and are subject to daily change based on market conditions and other factors.

Past performance is no guarantee of future results.

Top