Face Off: Siri vs. Google Assistant vs. Cortana

The importance of voice assistants in the screenless future is hard to overestimate. We see speech-driven user interfaces as a key component of the next computing paradigm, so it’s helpful to understand empirically where each platform stands today. In February, we compared Amazon Echo and Google Home to see which assistant was winning the race to become the centerpiece of the home. Google Home won that battle, but it was close. For our next digital assistant face off, we tested the three most prevalent digital assistants available for mobile devices: Siri, Google Assistant, and Cortana. This time, Google Assistant came out on top.

Methodology

We asked the same 800 queries to each assistant that we asked Amazon Echo and Google Home. We graded the queries on two metrics: First, did the assistant correctly understand the query? Second, did the assistant answer the query correctly?

The queries break down into five categories:

  • Local – Where is the nearest McDonald’s?
  • Commerce – Where can I buy more printer paper?
  • Navigation – How do I get to REI from here?
  • Information – What is Apple’s stock price?
  • Command – Remind me to call Mom at 2pm today.

Results

Google Assistant, the clear winner, understood 99.9% of the queries we asked and answered 74.8% of them correctly. Siri understood 94.4% of the queries we asked and answered 66.1% of them correctly. Finally, Cortana understood 97.3% of the queries we asked and answered 48.8% of them correctly.

By category, Google’s lead in navigation and information is demonstrable, but there’ more parity between Siri and Google Assistant in local, commerce and command related queries. Cortana lagged both Siri and Google Assistant in all categories, narrowly coming in second only in information.

Read More

Amazon Is Doubling Down on AI

Amazon’s Q1 earnings call last night sounded somewhat similar to Google’s call just an hour earlier, highlighting a lot of the same AI and machine learning investment themes.  Jeff Bezos discussed increasing leverage of AI and machine learning to drive operational efficiencies both internally and with AWS customers.  On a smaller scale, the beta Amazon Go store in Seattle is one of several competing visions for the future of retail. Leveraging computer vision, sensor fusion, and deep learning, Amazon Go allows for a checkout-less shopping experience.  We suspect Amazon’s bookstore expansion, beyond its six stores today, may soon begin to resemble characteristics of Amazon Go.

The most immediate AI lift for Amazon is coming via increases in Prime member engagement from Echo products and, more recently, Alexa-enabled Fire tablets.  Commentary from the earnings call indicated Prime Music, Prime Now, and Amazon Fresh Grocery are all seeing meaningful lift from those voice-enabled devices.  We highlighted Alexa’s recent momentum in our work, Can Anyone Catch Alexa?, now with more than 12,000 skills (apps) and roughly 100 smart-home manufacturing partners integrating Alexa.  Contrast this with, by our count, only around two dozen device manufacturers using Google Assistant.  With Bezos indicating a “doubling-down” on investment in Alexa and Echo last night, we expect Amazon and Google to remain head-to-head for leadership in the digital assistant market.

Another example of Amazon extending it’s lead in AI and machine learning is the new Echo Look, an Alexa-powered device for your closet. Echo Look features a camera that takes pictures of your outfit, then uses AI to make fashion recommendations and catalog your wardrobe.

Source: Amazon

Echo Look shows us what screen-less computing will look like in the future and, at the same time, how early we are in the transition. Amazon is pushing speech-driven computing faster and further, but the Echo Look also reminds us that we’re in the first inning of devices that will be increasingly commonplace as the next computing paradigm emerges. Adding cameras, sensors, computing power, skills and form factors to devices like the Echo product line will dramatically improve the utility of the category and drive adoption. Clearly, Amazon gets it and is determined to lead the way.

Disclaimer: We actively write about the themes in which we invest: artificial intelligence, robotics, virtual reality, and augmented reality. From time to time, we will write about companies that are in our portfolio.  Content on this site including opinions on specific themes in technology, market estimates, and estimates and commentary regarding publicly traded or private companies is not intended for use in making investment decisions. We hold no obligation to update any of our projections. We express no warranties about any estimates or opinions we make.

Can Anyone Catch Alexa?

Amazon’s third-party developer strategy has Alexa looking like the Gingerbread Man of digital assistants. After taking over CES in January with multiple Alexa-enable devices, there’s been little pause in Alexa skills and device-enabled growth. As of this writing, we count more than 12,000 Alexa skills (apps) and roughly 100 manufacturers with integrated Alexa IP across multiple smart home categories. And yesterday, Amazon opened its Echo voice processing IP to third party developers, extending Alexa’s lead in smart devices. Given our belief that natural language processing is one of a few core technologies that will enable the screen-less future of computing, we think it’s important to track the pace of the key players in the field.

Source: Amazon

Skills growth impressive, but getting the basics right remains the key for scale.

One measure of Alexa’s increasing utility is the growth in skills that can be downloaded to Echo devices. In just the last 3 months, nearly 5,000 skills have been added to Alexa’s repertoire, which now tops 12,000. Roughly a third of Alexa’s skills are knowledge-based – from education apps like the Old Farmer’s Almanac to trivia categories like Lesser Known Star Wars Facts. Other growth categories include health and fitness with skills like answers to common medical questions and workout suggestions. In addition, nearly 100 smart home skills are available today, an important catalyst for scaling the Alexa-enabled device ecosystem. It’s too early to tell how much scalable utility these skills bring to Alexa usage, particularly the nearly 500 knowledge/trivia skills categories. There is a fun-factor with a lot of these skills and voice access is seamless relative to paging through dozens of apps on your phone. However, Alexa needs to get better at answering basic information-related queries, which we believe will produce sustained utility and growth in Alexa-enabled devices. In a recent test, we found the Echo answered only 41% of information queries correctly.

We found the Echo answered only 41% of information queries correctly.

Device integration pacing well ahead of Google Home.

We count close to 100 manufacturers across several categories that are compatible with Alexa IP today. Smart home coverage, perhaps the most seamless hands free utility Alexa offers, continues to grow – from lighting to locks to thermostat control. Included on this list is Google’s own Nest device, a collaboration that began early last year; however, the relationship has been anything but seamless as a preponderance of Nest skill reviews suggest. We wonder if after a year of collaboration, whether Alexa and Google will ever nest together. Contrasting Echo’s ~100 smart home partners with Google Home, we find a much shorter list. Only around two dozen device manufacturers are integrated with Google’s Assistant IP. Amazon has taken advantage of Echo’s head start on Google Home by pushing integration across many manufacturers and platforms. Google Home will likely follow, but has a long road ahead.

Read More

The Five Senses of Computing

The trend in computing towards more natural user interfaces is unmistakable. Graphical user interfaces have long been dominant, but machines driven by more intuitive inputs, like touch and voice, are now mainstream. Today, audio, motion, and even our thoughts, are the basis for the most innovative computer-user interaction models powered by advanced sensor technology. Each computing paradigm maps to one or more of the five human senses; exploring each sense gives us an indication of the direction in which technology is heading.

Sight – Graphical User Interface

The introduction of the graphical user interface (GUI) drove a step function change in computers as productivity tools, because users could rely heavily on sight, our dominant sense. The GUI was then carried forward and built on with the advent of touchscreen devices. The next frontier for visual user interfaces lies in virtual reality and augmented reality. Innovations within these themes will further carry forward the GUI paradigm. VR and AR rely heavily on sight, but combine it more artfully with other inputs like audio, motion, and touch to create immersive interfaces.

Touch – Touchscreen Devices

PCs leveraged basic touch as a foundational input via the keyboard and the mouse. The iPhone then ushered in a computing era dominated by touch, rejecting the stylus in favor of, as Steve Jobs put it, “the best pointing device in the world” – our fingers.  Haptics have pushed touchscreen technology further, making it more sensory, but phones and tablets fall well short of truly immersive computing. Bret Victor summarized the shortcomings of touchscreen devices in his 2011 piece, A Brief Rant on the Future of Interaction Design, which holds up well to this day.

More fully integrating our sense of touch will be critical for the user interfaces of the future. We think that haptic suits are a step we will take on the journey to full immersion, but the best way to trick the user into believing he or she is actually feeling something in VR is to manipulate the neurochemistry of the brain. This early field is known as neurohaptics.

Hearing – Digital Assistants & Hearables

Computers have been capable of understanding a limited human spoken vocabulary since the 1960s. By the 1990s, dictation software was available to the masses. Aside from limited audio feedback and rudimentary speech-to-text transcription, computers did not start widely leveraging sound as an interface until digital assistants began to be integrated into phones.

As digital assistants continue to improve, more and more users are integrating them into their daily routines. In our Robot Fear Index, we found that 43% of Americans had used a digital assistant in the last three months. However, our study of Amazon Echo vs. Google Home showed that Google Home answered just 39.1% of queries correctly vs. the Echo at 34.4%. Clearly we’re early in the transition to audio as a dominant input for computing.

Hearables, like Apple’s AirPods, represent the next step forward for audio as a user interface.

Read More

Feedback Loup: College Panel

We recently hosted a panel of 8 college students from the University of Minnesota. The goal was to better understand how millennials think about social media, communications, video, VR, AR, the selfie generation, the future of work, and privacy. Here’s a summary of what we learned:

Text Is Dying

  • Quote: “Texting replaced email, and photos have replaced text messages”.
  • Message: Text is being used less frequently by each of our panelists. They view text as a formal way to communicate. Snap, Facebook and Instagram are the preferred communication platforms, with Facebook settings being switched to photos only. The panelists mentioned tech platforms promoting messaging within games as a way to maintain usage.
  • Takeaway: Text is slowly going away, replaced by video and photos. Text is viewed more as a formal way to communicate.

Fake News

  • Quote: “I like Snap for news.”
  • Message: Our panelists get their news from a wide variety of sources. 7 of 8 panelists are not concerned about fake news. Snap was the most popular way to aggregate news from traditional sources (3 of 8), followed by mainstream news outlets; e.g., CNN and WSJ.
  • Takeaway: Professional news is still respected but not paid for by these college students.

The Future of Work

  • Quote: “It’s scary. If we can’t have cashiers, truckers and fast food jobs. . . how will people live?”
  • Message: College students know they are entering a workforce that will have dramatic changes over the next 30 years. They have concerns about who’s going to control everything as resources become more concentrated. The University of Minnesota offers a class titled “Size of the Future” that addresses the risk of job loss to automation. The group did consider these changes when thinking about a career, with an increased interest in a more technical education that feels more defensible. Ultimately these students believe that the negative impact of lost jobs will be partially offset by the positive impact of new industries being formed.
  • Takeaway: College students understand that the workforce is changing. They envision social challenges emerging from displacement of workers with lower levels of education. But they believe a college education will ensure that their futures are safe.

Read More