Face Off Part 3 – Echo vs. Google Home vs. Cortana

We tested the Amazon Echo, Google Home, and Harman Kardon Invoke (powered by Microsoft’s Cortana) smart speakers, each with 800 queries, and found that Google Home answered 81% of the questions correctly vs. Echo at 64% and Cortana at 56%.

Did Santa bring you a smart speaker? Smart speakers are quickly becoming one of the most common consumer uses of AI thanks to aggressive pricing and marketing from companies like Amazon and Google. We’ve conducted our latest round of queries, designed to test the full range of abilities and accuracy of a given AI assistant. Our conclusion: home assistants aren’t just gaining popularity, they’re making drastic improvements quickly. Here is what you can expect from the smart speaker that you unwrapped yesterday.

Methodology. Just as we have in February and August of this year, we asked 800 questions to the Amazon Echo (2nd generation this time around) and Google Home. We added the Harman Kardon Invoke speaker, coupled with Microsoft’s Cortana assistant, to our evaluation. The queries covered five categories: Local, Commerce, Navigation, Information, and Command. The smart speakers were graded on two metrics: did the device understand what was asked? (this can be seen on the device’s companion app), and did it answer or execute correctly? It is important to note that we have slightly modified our question set to be more reflective of the changing abilities of AI assistants. As voice computing becomes more versatile and smart speakers become more capable, we will continue to update our question set to be reflective of those improvements going forward. Our changes included questions around the use of smart home devices. We tested each of the speakers with the Philips Hue smart lighting and Wemo Mini smart plugs.

This round, Google Home was the decisive winner, answering 81% of questions correctly vs. the Echo’s 64% and the Invoke’s 56%.  On average, the speakers understood what was being asked 99% of the time. This is a jump up from 95% in August and means that these assistants’ natural language processing has improved to correctly understand, apart from a few anomalies, everything you say. Whether or not it correctly executes your request varies widely between devices and categories.

Results. Google Home continued its outperformance with the top score in each of the five categories. The Invoke, powered by Microsoft’s Cortana, scored more or less in line with the Amazon Echo – an impressive feat considering its tiny market share and comparatively short time in the hands of users. The Echo made considerable improvements in several categories, most notably navigation, while remaining relatively flat in areas like commerce and local.

Improvement across the board. We continue to be impressed with the rate at which these AI assistants are improving. Since our first test in February, the total number of correct responses has increased by 29% for the Echo and 42% for the Google Home. Cortana, which we tested in April on a Microsoft Surface Book, has improved 8%. In less than a year we have noted remarkable improvement in every category and can only expect this level of progress to continue as adoption grows and user base expands. Google Home, however seems to be on the steepest improvement curve. One would expect Google to dominate the information category with its unprecedented access to search data, but its outperformance in categories like commands, navigation, and even commerce furthers the case, in our minds, for Google Home to steal market share away from Alexa going forward.

New features emerge. During our testing, we noticed several minor, but encouraging enhancements to the voice computing experience, namely connectivity to both other devices and other services. Excluding the Echo, when you ask for navigation information, Google and Cortana will send directions to your phone either through a push notification or the companion app. Hailing a ride through Uber was smooth on both the Echo and Google Home. The Google Home could also hand you off to a designated EBay assistant to shop for used goods. We were impressed with improvements around media services like Spotify and smart home capabilities using both lights and smart switches.

Google takes a commanding lead in commerce. Google Home correctly answered 72% of commerce questions, as opposed to 42% and 15% for the Echo and Invoke respectively. This is an area that is thought to be dominated by Amazon. However, Google’s recent anti-Amazon partnerships with retailers like Target and Walmart to make voice-based commerce more accessible pushed the Google Home ahead of other assistants. This was the Google Home’s largest area of improvement, correctly answering 34% more queries than it did in August. 

Sound quality. Overwhelmingly, smart speaker owners use them to call up and listen to music. According to Quartz data, 74% of owners play music through their smart speakers, more than asking for weather and basic info. Of the devices we tested, the Harman Kardon Invoke sounded the best; the Google Home and Echo are very similar, but are not focused on premium sound quality. For this mid-tier price range (sale prices – Google Home: $79, Echo: $79, Invoke: $99) the Invoke sounds best. Great sounding music, however, is less defensible than the AI itself. A useful assistant can be loaded onto a tiny speaker like the Echo Dot and Home Mini, or a premium speaker as we have seen with the Alexa-enabled Sonos One, Google Home Max, and the incoming Apple HomePod. It becomes clear that you are paying almost exclusively for sound quality.

Still not mainstream. Despite our comprehensive question set, the everyday user rarely extends beyond simple queries like asking for the weather, general knowledge questions, or requesting a song. In fact, 65% of Alexa users have not even enabled a skill. In order for voice computing to take hold in the home and beyond, there needs to be improvement in terms of connectivity with other devices like phones, laptops, and televisions. We did note much better connection to your phone via companion apps, your television via devices like a Chromecast of Fire TV, and better control of smart home applications, which is an encouraging sign. A short list of persistent issues keeps smart speakers in the camp of tech gadget instead of essential device (like a phone or computer), but at the rate of change we are observing, we remain confident that AI assistants, in time, will become an integral part of how we interact with computers.

Disclaimer: We actively write about the themes in which we invest: artificial intelligence, robotics, virtual reality, and augmented reality. From time to time, we will write about companies that are in our portfolio.  Content on this site including opinions on specific themes in technology, market estimates, and estimates and commentary regarding publicly traded or private companies is not intended for use in making investment decisions. We hold no obligation to update any of our projections. We express no warranties about any estimates or opinions we make.

2018 Market Outlook: Who’s Winning in AR & VR?

Written by guest authors Lindsay Boyajian, Product Marketing, and  Zack Kadish, Professional Services at Conductor.

For the past few years, we’ve seen the buzz around AR and VR build. All the major players from Amazon to Apple to Snap have made investments in the space. Funding for VR and AR startups, according to TechCrunch, has amounted to $2.1 billion this year in at least 217 companies in the AR and VR space.

However, is it all just hype? Is consumer interest really growing? Which brands and publishers have the most market share?

This report examines organic search data from Conductor to realize insights into consumer behavior and brand market share. The Conductor Searchlight platform tracked thousands of the most relevant, high-volume search terms that consumers used to find products, services, and information in Q4 2017 related to AR and VR.

Search market share indicates which brands and publishers own the top positions in search results. This data is invaluable because search behavior is a strong indicator of consumer buying intent. It provides insight into what brands, products, and services consumers are interested in.

The insights in this report highlight trends in AR and VR that will influence growth and adoption in 2018. It has category breakdowns to provide a better understanding of who’s dominating by product category and purchase journey stage.

Is AR or VR winning?

For years, VR was the rising star, overshadowing its cousin AR. But the tides turned in 2016 when experts began hailing the potential of AR. In September 2016, Apple CEO Tim Cook said, “My own view is that augmented reality is the larger of the two [VR & AR], probably by far, because this gives the capability for both of us to sit and be very present talking to each other, but also have other things visually for both of us to see.”

There are 30 times more searches related to virtual reality than augmented reality per month.

Despite the changing tides, VR is still leading among consumers. There are 30 times more searches related to virtual reality than augmented reality per month. Searches related to VR gaming are dominant, indicating that is where consumer interest lies today.

Search terms related to business applications of both VR and AR are negligible. For instance, “virtual reality in architecture” has a monthly search volume (MSV) of 320 and “augmented reality for healthcare” an MSV of 10. Neither technology is winning when it comes to owning the customer mind share for enterprise applications.

Which brands and publishers control organic market share?

Who owns market share for search terms related to VR & AR?

Who owns market share for search terms related to VR?

Who owns market share for search terms related to AR?

When we look at tracked searches related to virtual reality, publishers are dominant with some retailers, related to gaming (Playstation, Amazon, and Steam), capturing market share as well.

In comparison, publishers exclusively dominate tracked searches related to augmented reality. There are no retailers that have significant market share.

We also find that the majority of available market share is still unclaimed, represented by the large “other” slice. This indicates opportunity for late entrants to capture organic market share.

Are consumers ready to invest in AR & VR?

There are 5 times as many searches related to early stage intent compared to late stage. The searches are both more informational and comparative based (Early Stage and Middle Stage), rather than transactional (Late Stage). This suggests consumers are information gathering, rather than looking to buy a product or solution.

If we look at each technology individually, there are more transactional terms related to virtual reality than augmented reality. This indicates again that VR is winning when it comes to customer purchase intent.

What strategies can brands and publishers use to capture unclaimed market share?

Brands and publishers should consider optimizing their content for universal result types. 41% of all the tracked searches have video results. Given that most customers are searching for early stage content, brands and publishers should create early stage video content that answers questions like “how to” and “what is”.

Similarly, 30% of all the tracked searches have Google answer boxes, suggesting brands and publishers should optimize their content for answer boxes. Answer boxes are the search result features that appear at the top of the organic results in a Google Search.

When it comes to what topics to write about, the search data suggests there is opportunity to create content around transactional (Late Stage) search terms. These queries have relatively high search volume, but not a lot of high quality content relating to these topics. Creating content answering transactional queries is recommended.

Finally, as we have seen, publishers outperform manufacturers and brands in most categories— suggesting that it’s an ideal entry point for other brands and retailers. By contributing content or advertising on publishers that have high visibility, brands can also make a play to win market share.

The organic search data in this report comes from Conductor. For more, follow Lindsay Boyajian and Zack Kadish on LinkedIn.

Disclaimer: We actively write about the themes in which we invest: artificial intelligence, robotics, virtual reality, and augmented reality. From time to time, we will write about companies that are in our portfolio.  Content on this site including opinions on specific themes in technology, market estimates, and estimates and commentary regarding publicly traded or private companies is not intended for use in making investment decisions. We hold no obligation to update any of our projections. We express no warranties about any estimates or opinions we make.

Apple Wearables Drafting on Tech’s Push into Healthcare

Healthcare is in the midst of a dramatic transformation. This may seem obvious, but the culmination of this week’s news – CVS buying Aetna to create a new healthcare platform and Apple partnering with Stanford to carry out a medical study on AFib using the Apple Watch – brought the pace of change into perspective. Healthcare is transforming before our eyes, and new players are moving into the space that accounts for 18% of U.S. GDP.

Taking a step back, healthcare 100 years ago was fundamentally the same as it is today. We go to the doctor for two reasons – we’re sick, or it’s time for an annual checkup. The effectiveness of this approach is dreadful, illustrated by the fact that about half of all Americans have one or more chronic condition, diabetes and heart disease are on the rise, almost 40% of us are obese, and 7 out of 10 deaths are attributed to chronic diseases. All the tech advancements we’ve made have not kept Humpty Dumpty together. The reason for this is that healthcare is generalized, impersonal, and reactive in nature. The individual must fight the day to day battle of preventative care, not the provider who the average American sees only 4 times per year.

Today, we see a shift toward two themes – personalization and prevention – and the future of healthcare will be grounded in the frequency of health monitoring. CVS and Aetna are coming together to create what CVS CEO calls the country’s “front door to healthcare,” because more doors means more frequent access to care. Apple and Stanford aim to collect data on more people more frequently. The concept of increasing health monitoring frequency holds the greatest promise of actually making people healthier and the easiest approach to increasing frequency is through wearables.

Today wearables are seen as a luxury gadget for geeks and health nuts. In the future (7-10 years from now), we will be inseparable from our wearables, similar to our current obsession with smartphones. Today, the smartwatch is the wearable of choice. Soon, however, that could include things like hearables (think AirPods), contact lenses, and connected fabrics.

Driving with your eyes closed. Healthcare monitoring today is comical. Nootrobox CEO Geoffrey Woo on an a16z podcast put it into perspective by saying “imagine that we’re driving cars and we only let ourselves open our eyes every minute. That’s essentially the snapshot of information we get when we go to the doctor.” We go in for a checkup, make a course correction, then drift back into our old habits until the next time we see the doctor. Continuous health measurement is the most effective approach to stay our course corrections. We now have biometric sensors in common devices and the computing power to make sense of that volume of data. The benefit of continuous health measurement is twofold – it allows for large-scale data collection from which AI algorithms can derive insights, and it keeps your health top of mind. And it appears to be working, studies show that 70% of Apple Watch users track their heart health, even weeks after purchasing the device.

Apple’s got a tiger by the tail. Investor opinions on the Apple Watch range from “it’s a rounding error” (4% of overall revenue), to “it’s a dud.” The reason is investors had been spoiled by Apple’s vertical growth in new product categories with the iPod, iPhone, and iPad. Apple Watch simply didn’t live up to its predecessors. While it has been a slow start for Apple Watch, we believe the Watch and future (7-10 years) wearables (notably hearables) will account for a material part of future Apple revenue. As the health advantages of wearables begin to resonate, we foresee Apple selling as many them (Apple Watch and hearables in the future) as they do iPhones. At a wearable ASP of $300 (below current Apple Watch ASP assumption of $450) and 250M units a year, that would equate to $75B in annual revenue (not in our model today).

Apple has been engaged with the concept of healthcare since it introduced the Apple Watch in 2015, releasing ResearchKit that year and CareKit in 2016. While their new Heart Study is technically their first true medical study, the Watch has been used in the past for similar crowdsourcing of biometric data (along with Fitbit and others). So this begs the question, why is Apple interested in healthcare? Their core competencies are well-aligned to benefit from the shift toward personalized and preventative care. They also have a platform in their device user base and software frameworks, the data and AI power to carry out large-scale operations, and the design expertise to integrate sensors into devices that consumers want to use.

A word on hearables. Apple has tipped their hand. Earlier this year, the company filed patents suggesting AirPods may have a future as in biometrics. The patents outlined the addition of a photoplethysmogram, or PPG sensor, that can measure heart rate, VO2, galvanic skin, EKG, impedance cardiography, and temperature. We don’t have enough details to guess when these features might be integrated into a product, but do see a future when these hearables are continuously worn, giving users volume control of the world, as well as next-level, real-time health monitoring.

What about other tech companies. Don’t forget about Google and Amazon. One of Google’s other bets, Verily Life Sciences, is focused squarely on making healthcare more preventative and data-driven. Verily argues, “a new car has up to 400 different sensors. You know the oil pressure and how much air is in your tires, but we don’t do that with people.” They have undertaken an array of different projects from glucose monitors in contact lenses to eradicating vector-borne diseases by engineering and releasing fertile mosquitos. Verily’s efforts are largely complementary to Apple’s health ambitions, and their engagement in the space is confirmation that big tech companies have a place in healthcare. The opportunity here is substantial enough to accommodate more than a few entrants. While it is unlikely that Amazon will be a player in data or wearables, the company has the DNA to reinvent the logistics around how care is delivered.

It will take time to win over the “I don’t want to be monitored” segment. It’s going to take years for widespread adoption of health monitoring wearables, as defined by more than a billion daily users. As a point of reference, we’re at 40-50m today, with about 25-30m of those being Apple Watches. Some people resist continuous monitoring on the grounds of privacy, inconvenience, and anxiety around knowing their true health. That said, the resistance group will shrink over time (some due to poor health).

Disclaimer: We actively write about the themes in which we invest: artificial intelligence, robotics, virtual reality, and augmented reality. From time to time, we will write about companies that are in our portfolio.  Content on this site including opinions on specific themes in technology, market estimates, and estimates and commentary regarding publicly traded or private companies is not intended for use in making investment decisions. We hold no obligation to update any of our projections. We express no warranties about any estimates or opinions we make.

Alexa First to Business but Google AI Close Behind

Bottom Line. Alexa has 70-80% smart speaker market share and is pedal to the metal in expanding the market beyond the consumer today, being the first company to target the business opportunity. While we expect Alexa will continue to be the market share leader, we believe Google will close the gap in the next 3-5 years as AI becomes foundational to the future of smart speakers.

News. Amazon announced Alexa for Business today. The basic idea is Amazon is building skills around everyday business activities like conference calls, scheduling meetings, keeping track of tasks and ordering supplies. This includes integrations with Salesforce, Concur, SuccessFactors, Polycom and Crestron to name a few. They are also selling a hardware starter bundle which includes 3 Echo’s, 2 Dots and 2 Echo Show’s.

Bravo Alexa. Today’s Alexa for Business announcement is further evidence Alexa is leading the smart speaker market. Alexa is the market share leader for good reason. First, Alexa is smart, and now has over 25k skills compared to 12k that we counted in April (Google does not have skills rather actions).  Second, Alexa 3rd party hardware integration is expanding and earlier this year we estimated there were about 100 manufactures with integrated Alexa IP.

AI, the elephant in the smart speaker room. While it may look like Alexa is running away with the smart speaker market, Google is gaining ground. In October at the Google Hardware Event, Google explained how hardware products will facilitate AI first computing. In 2017 CEO Sundar Pichai has opened each of his public remarks stating Google’s goal of becoming an AI first company. This has obvious implications for Google’s advertising, Maps, YouTube, cloud business and now hardware.

Google likely to gain smart speaker share in 3-5 years, but Alexa will still be a share leader. Google’s efforts in the next few years could yield a measurable increase in market share. As mentioned, today we estimate various forms of Alexa account for roughly 70-80% smart speaker share and we envision Google’s share increasing from about 25% today to greater than 35% in the next 3-5 years.

Why Google’s in a good position. While Google is lightyears behind Alexa’s install base, we believe Google has the best AI (see our comparison here), and their more robust product line could catch up quickly. Google is going toe-to-toe with Alexa in terms of hardware pricing (Echo Dot and Google Home Mini starting at $29). More importantly, we expect Google will weave their AI Assistant into the fabric of the device ecosystem. This is important because integrating an array of devices (i.e. the handoff between the home and the road) is what will push users toward the next generation of our interaction with machines.

Disclaimer: We actively write about the themes in which we invest: artificial intelligence, robotics, virtual reality, and augmented reality. From time to time, we will write about companies that are in our portfolio.  Content on this site including opinions on specific themes in technology, market estimates, and estimates and commentary regarding publicly traded or private companies is not intended for use in making investment decisions. We hold no obligation to update any of our projections. We express no warranties about any estimates or opinions we make.

Waymo Unleashes Autonomous Cars – Now Must Earn Public’s Trust

On November 7th, Waymo reached a new milestone by removing the safety net and testing fully autonomous vehicles on public roads without someone behind the wheel to take over in case of an emergency. Waymo has been testing in the Phoenix, AZ area for some time now, and other companies like Uber, Cruise, and NuTonomy have similar operations. But there has always been an employee in the driver’s seat. November 7th marked a new stage of testing self-driving cars. Along with last Tuesday’s test, which took place in Chandler, AZ, Waymo has recently made an exciting push to prepare the public for the cars that, as they just proved, are closer to full deployment than many people believe.

Largely considered the leader in autonomy, Waymo has driven a collective 3.5 million autonomous miles on public roads across 20 cities (that’s the equivalent of the average American driving for 291 years). They have also completed over 20,000 different scenario tests at their facility in California, and simulate 10 million miles per day. “In short: we’re building our vehicles to be the most experienced driver on the road,” they write in a blog post. Along with a growing crowd of other competitors, Waymo gets closer each day to deploying a fleet of self-driving cars available to summon at your convenience. Regardless of how advanced the tech may be, however, the reality remains that people simply aren’t ready to be driving down the road next to an empty car – perhaps the biggest hurdle facing autonomy is widespread acceptance of them.

Being realistic about autonomy. Waymo knows that in order for mass adoption to take place, the public must first trust autonomous vehicles. Successfully building a vehicle that can operate autonomously is a feat of engineering, but educating the public on its benefits is a different task entirely.

Realizing this fact, Waymo has recently made an impressive effort to prepare the public for what’s coming. They have partnered with organizations like the National Safety Council and Mothers Against Drunk Driving, trained law enforcement departments on how to deal with incidents involving self-driving vehicles, and attempted to be transparent by releasing a comprehensive Safety Report and inviting reporters to a test drive at their facility in Atwater, CA. According to a AAA survey this year, only 20% of Americans would trust an autonomous vehicle to drive itself with them in it. This leaves no doubt that we have a long way to go before this technology becomes mainstream. Waymo, more than any other player in the space, is attacking the problem head on, opening up a dialogue with the public and taking an inclusive approach to educating everyone on the risks and benefits of a new type of mobility.

Partnerships. By engaging the public and partnering with organizations outside of tech and auto, Waymo hopes not only to raise awareness and educate people on self-driving cars, but to demonstrate how they are, in fact, a much safer and smarter mode of transportation. Here are some of the groups that Waymo has teamed up with:

  • The National Safety Council – Focused on areas where the greatest number of preventable injuries or deaths occur, including workplace safety, prescription medicine abuse, teen driving, and cell phone use while driving. 40,000 Americans die on the road each year.
  • Foundation for Senior Living – Believes age shouldn’t slow anyone down. 80% of seniors live in vehicle-dependent suburbs, and there are 45M people in the U.S. over the age of 65.
  • Mothers Against Drunk Driving – Intoxicated driving is the number one cause of death on roadways.
  • The Foundation for Blind Children – Focused on empowering the blind with independence. There are 1.3 million legally blind individuals in the U.S., growing to over 2 million by 2050.
  • East Valley Partnership – Concerned with improving quality of business and life in the East Valley region. Americans spend 50 minutes on average commuting to and from work each day

Waymo’s Safety Report. “Fully self-driving vehicles will succeed in their promise and gain public acceptance only if they are safe.” This thesis, stated in the early pages of a recently released Safety Report, resonates throughout the next 43 pages, as Waymo lays everything on the table. The report details safety procedures, how vehicles respond to numerous situations, how the autonomous systems function, and several other elements that must be understood before feeling comfortable riding in a self-driving car. It quickly becomes clear that safety is at the core of Waymo’s pitch. As the first voluntary safety assessment of its kind, much of its contents will likely be mandated by regulatory bodies going forward.

Law Enforcement Training. Yes, there will still be accidents on the road when cars drive themselves. While we believe there will be radically fewer of them, law enforcement must still understand how to respond to an incident involving a driverless car. Waymo has designed their systems to interact with law enforcement and first responders with audio sensors to discern where sirens are coming from, and responses like safely yielding or pulling over to a complete stop. They have also briefed authorities in every city in which they test, and conducted on-site trainings to help police and emergency vehicles identify and access self-driving cars.

Test Drives. On October 31st, Waymo hosted a group of journalists at their usually secret testing compound in Atwater, CA. This act is not unprecedented; however, coupled with Waymo’s other recent actions, it represents a level of transparency unmatched by any of their competitors. The group was given a test drive in a mock town they have created, complete with an array of real-life scenarios like an unexpected cyclist cutting in front of the car, or a man standing beside a broken-down Hyundai. Find a detailed write-up of the test drive here or here.

The idea was to give riders a feel for what it’s like to use Waymo’s Chrysler Pacifica minivans as on-demand vehicles. It will function a lot like current ride-hailing platforms – a rider summons a car on a smartphone app, the car locates the rider, and navigates to their destination. Press a friendly, blue “Start Ride” button to embark, and passenger-facing screens show a rider-friendly version of what the car is seeing.

Trust building 101 – with transparency. As a cyclist rides by or a car passes, it appears on the monitor – it even shows trees, parked cars, and buildings in your surroundings. The level of detail that Waymo focuses on in terms of user experience leads one to believe they are months, not years, from deploying their much talked-about fleet. A video from Waymo exhibits how remarkably smooth the process is. Between a groundbreaking and successful test, and a new level of transparency focused on building trust and engaging the public, we believe Waymo has earned its pole position in the race for autonomy. Their next step will likely be the deployment of a small fleet to a limited group of participants in AZ – stay tuned.

Disclaimer: We actively write about the themes in which we invest: artificial intelligence, robotics, virtual reality, and augmented reality. From time to time, we will write about companies that are in our portfolio. Content on this site including opinions on specific themes in technology, market estimates, and estimates and commentary regarding publicly traded or private companies is not intended for use in making investment decisions. We hold no obligation to update any of our projections. We express no warranties about any estimates or opinions we make.