Liberating your Data with “Google-esque” Enterprise Search

(00:04:42.00)

Michelle:
What's changing now and what does it mean for us and our customers to address this issue?

Karl:
Well, the introduction of AI means that we're now able to build our own cognitive search experiences. Gartner and Forrester refer to the search space as insight engines and cognitive search respectively. And that's basically recognition that we've moved on from keywords to AI. It applies in two areas: firstly, enriching content as its index to make it more discoverable using things like image recognition, entity extraction, document understanding, and categorization. But then on the experience side, the ability to understand user intent, allowing us to ask for the information we want in natural language and returning more relevant results and actual answers to questions. We also see things like knowledge graphs to help add context and expand query terms just as Google does.

Michelle:
So you're saying that search experiences vary quite a bit and there's an opportunity to improve? So how do you measure that?

Karl:
So it's a great question, Michelle. The thing about search is that it's a mature technology and most organizations already have experience of a legacy search technology. So recognizing this, we're creating a maturity model to help our customers understand where their search experiences are today and how they can start on their journey to AI. So we identified four maturity phases, each with an organizational and product maturity component. And this is the key reason why you can't just go out and buy the best search product if your organization isn't ready for it. Great search is a combination of technology, expertise, and relentless measurement and optimization.

(00:06:24.03)

Michelle:
How do organizations move from phase one to phase two to phase three?

Karl:
Baseline search is typically a legacy platform or a default implementation of a built-in search engine, like in a CMS, a content management system. There's probably no real stakeholder at baseline maturity. It's usually pretty easy to progress to advanced by addressing the organizational data first. So assuming someone is incentivized to actually fix this, like senior management hearing all about how bad the search is, what I would do is get some advice on the capabilities of the existing platform and modernize it immediately if needed, then set up a search management team and identify a product donor. And their job is to work together to understand, measure, and improve the experience. It's really all about seeing search as never done as we both know, then it really should just be a natural and iterative process driven by analytics and direct feedback. And once you're at advanced and everything's working, well, that might be enough for some to stop there. Phase two to phase three is a bit harder. For this, you'll need to be on a modern search platform, so there could be some technology investment there. And this requires some high level stakeholders on the experience side to really wanna have the best-in-class search experience in your vertical. At this level, you'll definitely be using some of the new AI features such as natural language processing, with data and machine learning to drive relevancy. Someone also needs to be owning an ongoing vision for search and working with either your own development teams or a supplier to deliver that.

Michelle:
You and I have talked through this before, and you've said that organizations can't skip phases. Why is that?

Karl:
Well, as you progress through the maturity levels, you're basically understanding more about what your customers actually want from search, how they use it, and where you can take them to next. So it's a journey and whilst that can be accelerated now with the right technology and guidance, so we can help with that for example, for the most the organizational challenge means you have to work up to the next phase of maturity.

Michelle:
Things get really interesting in phase four. Walk us through that. What's reality there and what is still aspirational?

Karl:
When we first created a version of this model, about four years ago, the right hand side was a vision we'd come up with by connecting the dots across search and conversational UI. We tend to still think of search and chat as different front-end experiences. But on the back-end today, they're both about natural language processing and information retrieval. So why can't a customer today ask a question in the search box on a retail site, for example, do you have this in blue? When is this item back in stock? It doesn't have to be in a separate chat bot. So in 2016, when we did this, it was all a bit blue sky, but today products like Google's Cloud search and Dialogflow can bring together search and chat with the same AI technology. Other vendors, like Lucidworks, have AI-powered search as part of Fusion and have recently added a feature called Smart Answers. So you can basically see where it's all going.

Michelle:
One thing that I hear frequently from organizations is that AI can solve all of their search problems, but AI isn't a silver bullet, right?

Karl:
Absolutely not. Fortunately there are no AI magic ones here. Part of the issue with search for many years has been this idea that you just buy the best and it'll all work. But unfortunately, as we both know, search has never been like that. Whilst it can now be much more intelligent through AI, AI is not gonna help you with some of the more prosaic challenges such as connecting to data sources, data quality, implementing security models, and actually creating a frictionless and engaging user experience.

(00:10:04.09)

Michelle:
Where are you applying this directly? Can you give us examples of where you're seeing this kind of search modernization taking place and what you can do to get started? Let's start with in-app search and site search.

Karl:
So in-app search, I see a lot of scope to improve. That's on mobile. We've had a nice example in healthcare recently where an insurer is looking to provide better access to care. And this is a good example where a traditional search approach, which is not intelligent, just doesn't work. You've got issues such as members using consumer health terms like knee doctor, when the data's full of medical terms like orthopedic surgeon. And that is then compounded by the typing errors we get from the fat finger effect on a mobile keyboard. As we all know, most of what we type into Google on our phones can be all over the place. But as a solution with search, it can be done, and we should expect more of a typical in-app search feature today. Site search, I'm talking about your typical brand or corporate website search on the CMS, sort of not e-commerce. People are usually looking for information about products, services, company details, and the latest annual report. These kinds of sites have been underserved for years. They're commonly at a baseline level of maturity. We've worked on many of these and there's always an opportunity to create a great customer service channel by adding a modern search experience. And these sites are basically sending a really negative message to customers that we don't care through an existing poor search experience. I'd actually encourage sites to not have a search box at all, rather than a bad one.

Michelle:
Let's talk about retail next. The increase in online demand is seemingly making every day like Black Friday. What are you seeing there?

Karl:
Retail is perhaps the most interesting one right now, given all that’s going on with the pandemic. Most of us are well aware that retail has seen huge increases in demand online, and that also this is likely to continue. But that increased demand has just magnified all of the existing issues with search as well. So more zero results queries, more dissatisfied customers. So this is where there's a real opportunity to embrace a modern search platform and create something truly transformative. Retailers have a ton of behavioral data. So using that along with new AI search technology can really enable this next generation of search experiences in ways that massively reduce friction and improve convenience. The pinch point in search is always single word queries, where there are a gazillion results and you need some intelligent signals to rank them effectively. For example, if I'm a regular customer and I'm searching for something like coffee, I wanna see the kind of coffee that I buy first. In fact, even in the autosuggest dropdown, along with an add to cart button. I don't have to scroll through results to find it. In retail, merchandising is another classic pain point. So merchandising teams commonly have this hairball of hard coded rules to deal with poor search relevancy, but the intelligence is there today to fix a lot of this in the search platform. And that means merchandisers can actually get on with what they should be doing. If I was in charge of search in one of these retail sites today with a scaling or a search quality issue, I'd be seriously planning for the kind of modernization intelligence and elastic scaling that I'm talking about.

Michelle:
And what about B2B?

Karl:
Well, so much of B2B is actually behind retail. I would put this in the same category as retail in terms of the opportunity. But the potential positive impact on customers is huge because the overall maturity for B2B is typically lower than B2C in search.

Michelle:
Having worked alongside B2B organizations, I would definitely agree with you. The B2B space has so many opportunities as customers become more self service. Let's touch on enterprise search.

Karl:
Yeah, great one. So the first opportunity in the enterprise today is to use these new AI search platforms to unlock the value in all of your unstructured data so much more effectively than how things have been done in the past with legacy tech. So it's about unveiling documents, reports, research, finding people in your organization that can help you make better decisions, and just generally being more efficient, and not duplicating effort. Now with this shift to remote working we're all experiencing, employees don't have access to their coping mechanisms, the analog tools in the office. They might use white boards or print outs, asking a colleague. They've all gone. So great search can make a huge difference here.

In terms of the state of search technology in the enterprise, we heard that 90% of enterprise search is still on legacy technology. That was actually researched from Bank of America. Merrill Lynch mentioned on Elastic's first earnings call back in 2018. And I doubt much has changed between now and then. So legacy search tech typically required a huge investment and had a high cost of ownership. A lot of it under delivered, there was a lot of disappointment because the teams were just not there to own and manage the deployments. So one thing I've seen consistently over the years is that few organizations have managed to maintain the maturity required to fully own their enterprise search platform and get value out of it. But with the cloud and AI, it's actually easy to modernize today and a lot of the heavy lifting can be removed. So that means you can have a smaller team looking after it. So whilst on the customer side, AI can create amazing omni-channel experiences in natural language, including search. In the enterprise, the bigger opportunity is to use AI-powered search as a great way to liberate your unstructured data, but to actually use this as a start on your journey towards more cognitive platforms, which inject intelligence and automation into your core business processes, decision making.