We Ran HomePod Through the Smart Speaker Gauntlet

Conclusion. Yesterday, we put HomePod through the smart speaker gauntlet which included 782 queries along with comprehensive tests of sound quality and ease of use. Our methodology was comprised of 3 HomePods being tested throughout the day by the Loup Ventures team. Here are four key takeaways.

  • HomePod is hands down the best sounding smart speaker.
  • As a smart speaker, it answered 52.3% of queries correctly compared to recent tests of Google Home at 81%, Alexa at 64%, and Cortana at 57%.
  • The user experience of HomePod is measurably better than its competition (setup, communication style, listening ability).
  • We have added HomePod to our Apple financial model. For CY18, we’re expecting HomePod unit sales of 7 million (12% global smart speaker market share) with a $349 ASP, which adds approximately 1% to revenue and earnings. In CY19, we expect unit sales of 10.9 million, which adds just over 1% to our model. While only factional revenue contribution, we expect HomePod units to grow between 40% and 45% per year over the next 3 years. Link to model here.

Query results. Siri understood 99.4% of queries and answered 52.3% of them correctly. This places HomePod at the bottom of the totem pole in terms of AI assistant performance. Siri is particularly strong in Local (where can I find a good cup of coffee around here?) and Commerce (help me buy some new shoes.) queries, handily beating Alexa and Cortana, but still falling short of Google Home in those areas. Overall, Siri performed above our expectations given the limited scope of HomePod’s music focus.

Adding domains will quickly improve Siri’s score. Some domains like navigation, calendar, email, and calling are simply not supported. These questions were met with, “I can’t ___ on HomePod.” Also, in any case that iPhone-based Siri would bring up Google search results, HomePod would reply, “I can’t get the answer to that on HomePod,” which forces you to use your phone or give up on the question altogether. Removing navigation, calling, email, and calendar-related queries from our question set yields a 67% correct response, a jump from overall of 52.3% correct. This means added support for these domains would bring HomePod performance above that of Alexa (64%) and Cortana (57%), though still shy of Google Home (81%). We know Siri has the ability to correctly answer a whole range of queries that HomePod cannot, evidenced by our note here. Apple’s limiting of HomePod’s domains should change over time, at which point we expect the speaker to be vastly more useful and integrated with your other Apple devices.

Other observations

  • HomePod has superior listening skills to other smart speakers. This is partly due to a noise cancellation feature which allows you, even at a volume where you would have to raise your voice to talk to others in the same room, to use your regular speaking voice with Siri. This was HomePod’s most stellar feature.
  • Wireless setup was super easy. In fact, it was the easiest of any of digital assistant we have used (Alexa, Google Home, Cortana, etc.). You already have a companion app and don’t need to wrestle with wifi networks.
  • Siri’s voice sounds smoother and more human than it does on your iPhone.
  • Her communication is also more human-like. Specifically, after asking a question, she does not repeat the whole thing back to you as is the case with Google Assistant and Alexa, which makes for a subtly smoother process.
  • The tap UI on the touch-sensitive display requires a small amount of instruction, as it is not immediately obvious that you tap to play.pause, double tap for next track, triple tap for previous track, and touch and hold to bring up Siri.
  • HomePod’s packaging is a new level of perfection for Apple. The perfectly fitted box requires you to open it at “reveal” pace, and even the external plastic wrap is pleasing to remove.
  • As a speaker, it sounds incredible – mission accomplished. As a digital assistant, whether it is a direct competitor or not, it is better than what we expected for version 1, but still lags behind Alexa and Google Home.

Survey suggests demand for HomePod similar to Apple Watch. Last week we surveyed 500 people in the U.S. and found 3.3% said they would purchase a HomePod in the next year. Among those surveyed who already own an Apple product, 5.2% planned on buying a HomePod. This is similar to 7% of Apple product owners planned to buy an Apple Watch ahead of its launch in Spring of 2015 (survey was conducted in December of 2014, four months ahead of the Apple Watch launch). While Apple does not disclose Watch unit sales, we estimate in its first 12 months Apple sold 10.2m Watches at an ASP of $475. This compares to our first 12 months of HomePod sales estimate of 7m units with an ASP of $349.

Smart speaker market share. We see Google Home as the long-term smart speaker unit share winner, but Alexa and Apple as the two other key players. In 2018 we expect HomePod will capture 12% of the global smart speaker units, compared to our estimate of Alexa at 52% share, Google Home at 32% and others at 4%. In 2022, we expect HomePod will hold a similar 12% market share (HomePod ASP estimate declines to $149 from $349 today), Google Home at 48% and Alexa at 37%.

Apple’s grander vision around HomePod.  Don’t be fooled by HomePod’s sound quality-focused first step into smart speakers; Apple has a grander vision than delivering a better sounding Echo. While not present in the first version of HomePod (i.e. you can’t even make a phone call with HomePod), we believe Apple’s goal is to make Siri a ubiquitous, ambient presence that connects and controls all your connected devices and services – and to make a leap forward in the transition to voice-first computing.

The way humans interact with computers is changing. Today, we use our keyboards, mice, and touchscreens to interact with computers, but in the future, we’ll simply rely on our voice, gestures, or even our thoughts. Voice is quickly becoming a preferred interface. Apple’s device ecosystem delivers a frictionless experience, which will only get better with the adoption of voice and with the addition of HomePod supported domains (9 supported domains today). Interestingly, Apple has included an A8 chip in its HomePod, the same chip included in an iPhone 6. The A8 chip is much more powerful than the chips competing home assistants run on, which poses the question: what else is Apple planning with the HomePod?

Disclaimer: We actively write about the themes in which we invest: artificial intelligence, robotics, virtual reality, and augmented reality. From time to time, we will write about companies that are in our portfolio. Content on this site including opinions on specific themes in technology, market estimates, and estimates and commentary regarding publicly traded or private companies is not intended for use in making investment decisions. We hold no obligation to update any of our projections. We express no warranties about any estimates or opinions we make.