The Robo Ranking™
The Robo Ranking™ was developed by Backend Benchmarking as a comprehensive analysis of the different robo advice providers.
Backend Benchmarking began opening accounts at robo advice providers in 2015 with the goal of bringing transparency to the robo advice industry. Although performance is an important component when assessing any investment manager, it is not the whole picture. Backend Benchmarking developed The Robo Ranking™ to help the investment community understand the products and services of robo advice providers beyond just performance.
The Robo Ranking™ incorporates our findings on portfolio construction, risk, and performance with a comprehensive analysis of the services and capabilities of these online platforms and ranks them against each other.
The scoring system that drives the ranking is broken into two main sections. The quantitative section scores the performance of the portfolio by using the Normalized Benchmarking method and the Sharpe ratio, a more traditional metric. Management costs and fees on the underlying investments also contribute to the quantitative score. The qualitative section dives into the services and online platforms of robo providers by scoring the platforms on more than 35 specific metrics and organizing them into six areas. These areas are: financial planning, user interface and customer experience, product features, access to live advisors, transparency and conflicts of interest, and account minimums.
As the only major provider of accurate returns data on robo portfolios, Backend Benchmarking’s Robo Ranking™ is an unrivaled resource for individuals interested in investing with a robo advice provider.
A detailed explanation of how we construct the ranking can be found below.
How We Rank the Robos
The robo advisors are ranked on a comprehensive set of criteria. The final robo score is made up of a qualitative score of their services, platform, and features, and a quantitative score based primarily on the costs and performance of the portfolio. A small portion of the quantitative score is based on the size, and tenure of the robo advice product. When looking at the qualitative aspects of the service, we focus on six categories: financial planning, user interface and customer experience, product features, access to live advisors, transparency and conflicts of interest, and minimum investment.
Below, we give examples of what earned points in each category.
Financial Planning: Here we graded the platforms on the quality of financial planning services offered. Robos that allowed users to build or create single or multi-goal financial plans were awarded points. Other financial planning tool features that earned points were those that allowed for “what if” scenarios; helped users calculate retirement spending needs, including social security benefit estimates; allowed for the inclusion of pension or other retirement income; and offered suggestions on appropriate monthly saving goals. In this issue of the Ranking, points were awarded if their planning tools had specific functionality. For example, if the single-goal planning tool could: One, model future account values or spending; two, accept a user input of an account value or spending goal; and three, show either a likelihood of success or changes to improve goal outcomes, then all points were awarded. If only some of these features were present, then partial points were awarded. Note that half points were awarded instead of full points if any of the following criteria were met: if the financial planning feature was only made available at a higher tier, for additional cost, or if the financial planning feature was not integrated with the digital portal.
User Interface and Customer Experience: Here we evaluated the user interface and the digital customer experience. We looked at the ease of getting to basic account information and general accessibility of the site. We measured the number of clicks required to access basic account and portfolio information, and used third-party software to produce an “accessibility score.” Points were also awarded to platforms that had good content and articles on basic personal finance and investing topics. During onboarding, we looked to see if the onboarding questionnaire took into account a user’s comfort with investing and inquired or mentioned whether the user has an emergency fund. We also scored robos that had the ability to aggregate held-away accounts for a holistic financial picture. Availability of live chat options and mobile apps also helped robos score higher in this category.
Product Features: Robos were awarded points for different types of features. Tax-loss harvesting, tax efficiency, tax location strategies, smart dividend reinvestment, ability to trade fractional shares, cash management features, types of accounts offered, access to impact or other themed portfolios, and the ability of a robo to customize a portfolio to a specific customer situation were the features we looked for in this category. We also included a field for unique and additive features that were not explicit in our scoring. This unique and additive features criteria was a small portion of the overall features score.
Transparency and Conflicts of Interest: In this category, we looked for things like whether or not a user could easily compare their portfolio to relevant benchmarks to help them understand performance. We also awarded points for platforms that made their models available before account opening or becoming a client, and further points if they also published the performance of their models publically to prospective customers. Availability of white papers and other information on how portfolios are constructed were also awarded points. We also awarded points to those portfolios that did not rely entirely on proprietary products or chose no proprietary products when constructing their portfolios.
Access to Live Advisors: Robos with access to live advisors, or the ability to upgrade to a product that has live advisors, earned points. Advisors need to be able to advise or provide financial planning guidance on customer-specific questions to score points. Live customer service and operational support is not sufficient for us to consider it a live-advice relationship. Robos earned more points if there was a dedicated live advisor option, if they required their advisors to hold CFPs, and the minimums at which live advisors are made available. Partial points were awarded to firms that had products or programs with live advisors if those programs were not part of the digital advice product suite, if they are offered at a higher service tier or for additional cost, or if live advice options require a high minimum account balance.
Account Minimum: Robos earned points for having lower investment minimums.
Costs: We scored costs on the sum of the management fee and average-weighted expense ratio rather than scoring these two components separately. This method better reflects the true cost incurred by clients. Additionally, we consider a cash allocation as a cost if the cash holding is earning less than a competitive rate which is set based on prevailing market rates for each ranking. The cash allocation had a much smaller impact than management fees and weighted expense ratios.
Performance: We used two metrics to grade a robo’s performance. The first was the Sharpe ratio, which is a measure of risk-adjusted returns. The second was their return above/below the Normalized Benchmark. This measurement method reduces the impact of different equity/bond allocations in the portfolio. The method of using a Normalized Benchmark was created by the team at the Robo Ranking and is explained in detail in the Normalized Benchmarking section on the website. The performance time period analyzed is consistent across all robos in each ranking.
Size and Tenure: This score is based on the AUM and age of the robo advice products. Large amounts of AUM and older products are less likely to be discontinued in the future, forcing a client to change providers or products, which can be disadvantageous to the client. Robos that do not publish their AUM specific to the robo advice product only received the points available for the age of the robo. We encourage robo advisors and their parent companies to release AUM data for their different products in the interest of transparency to the investor.