Optimising product search

The SAP Commerce platform allows for great flexibility in configuring the search engine it ships with out-of-the box, the popular open-source enterprise platform Apache SOLR. Though powerful, the search performance is dependent on how well the search was configured and optimised. A well configured search can give a boost to the product discovery process.

This blogpost seeks to explore some of the considerations, do’s and don’ts and a general strategy on how to build an effective search in a commerce environment, whether that’s SAP Commerce / SOLR or any other platform configuration.

Facet search

Searching in a commerce environment is often combined with faceting, which dictates a specific strategy. Whereas a google search might yield 100.000 results with the most relevant results at the top, a facet search allows the users to apply filters based on facets of the results. A product usually has a name, but also often a brand, a category, maybe a size etcetera, all of which are facets which can be filtered on. A customer might be shopping for men’s shoes, which are sneakers, white, in size 8.5, of brands A, B and C.

The objective here is reducing the size of the search result as fast as possible, going from a large result of maybe 100.000 products to a result set that’s feasible to browse through as a customer, a couple of product list pages filtered for relevance.

It can be compared with the game “Guess Who?” where you might open with a question like “is your person male or female?” to be able to knock down about half of your cards. Because of this, it doesn’t make sense to show all possible facets from the get-go, because there might be a lot, and they might include facets that only apply to a small subset of products, such as ‘colour of laces’ or ‘with detachable collar’.

Thus, in the example of the apparel store, you might want to show the facet ‘men’ or ‘women’ first after a search on ‘shirt’, followed by maybe brand, or colour. Facets such as ‘longsleeve’ or ‘shortsleeve’ or ‘collar type’ can become relevant when drilling down into that search result. Facets become irrelevant when they don’t reduce the result anymore.

With a large and diverse product catalog, and a long list of possible facets, it can be valuable to have a mechanism in place to rank, sort, show and hide facets based on their potential to reduce the search result.

Improve the data

As a search engine links a query to a product based on its data, there can be a lot to gain in improving the product data set. Take the following example:

• Product 1
  • Name: ST1000 4K
  • Category: Monitors
• Product 2
  • Name: Monitor stand for ST1000 monitor
  • Category: Monitor accessories

The user that searches for “Monitor” might get tons of nuts and bolts meant for monitors as top results, but not monitor itself, because the search term matches on a lot of properties of products that are essentially accessories or otherwise related.

This can be solved by adding another field that search terms can match on, which describes what products actually are, and boosting this field in the search results. For example:

• Product 1
  • Name: ST1000 4K
  • Functional name: Monitor
  • Category: Monitors
• Product 2
  • Name: Monitor stand for ST1000 monitor
  • Functional name: stand
  • Category: Monitor accessories

In SOLR, it’s possible to assign a boost to a certain field, so in the above example, a match on functional name would rank higher in the search result compared to others.

Synonyms and keywords

Using synonyms and keywords in the right way can improve search results, while using them in the wrong way can cause a lot of noise. Synonyms are a two-way street, such as TV and television. Searching on TV will match documents that contain the word television. Keywords are a one-way-street however, as an apple is fruit, but not all fruit are apples.

Once keywords are mistaken for synonyms and fed as input to the search engine, there’s a lot of potential false-positives. It can be fruitful to critically review synonyms and keywords to see if they’re set up the right way.

The waterbed problem

One of the pitfalls of search optimisation is that optimising the result of a certain search query may very well negatively impact another. For example, one might configure a wildcard search so that users can find a product by its product code, or a partial product code. Searching on 71812492 will find product 71812492, but 718* will also match. In this case, the sequence 718 might also match on a lot of other totally unrelated products based on properties such as model number, height, name etcetera, causing unwanted noise in the search result.

Search optimisation means always walking the tight rope between showing too much, which can overwhelm the user, and showing too little, which can obscure relevant hits from the user. How to mitigate against the waterbed problem? Compile a list of the most common and valuable search queries and optimise for those. Then use that list as a test script to periodically (or after any changes) check if those important searches are performing well. Do so in the knowledge that there’s always going to be search queries that yield a result that is less-than-satisfactory.

Final notes

Optimising search results is a science in its own right, and it’s important to recognise this when setting out to enhance your search engine. While there are vast amounts of resources available on this topic, knowing which approach to take in what situation takes experience and domain knowledge acquired through in-depth involvement in different projects. There are a lot of variables at play, which can include (in a commerce environment) the specific type of product, customer or commercial strategy of the platform.