Sorting by 'Price + Shipping: lowest first' shows the wrong result

[gazarcher]

I've read a few messages in the Community forum and it looks like sorting after doing a search has been a problem over the years.  I've experienced a sorting bug for weeks, maybe months now and I'm finally going to ask about it / report it.

Here are the steps:

From 'My eBay' enter '(britains,deetail)' in the search box, select 'All Categories' in the drop-down menu and  click the 'Search' button.
Result: 16,744

Click the 'include description' checkbox.   Under 'Categories' select 'Toys & Hobbies'.
Result: 13,657

Click the 'include description' checkbox.   Under 'Category' select 'Toy Soldiers'
Result: 8,412

Click the 'include description' checkbox.   Under 'Category' select 'Pre-1970'
Result: 1,534

Click the 'include description' checkbox.   Under 'Item Location' select 'US Only'
Result: 1,206

Click the 'include description' checkbox.  In the sort drop-down menu, select 'Price + Shipping: lowest first'
Result: 177

Huh?  What happened to the other 1,029?

In the sort drop-down menu, select 'Price + Shipping: highest first'
Result: 1,224

It's not exactly the same, but allow for the dynamic nature of eBay when items are being added and sold at the same time.

Try 'Price + Shipping: lowest first' again and I get 177 again,  Try 'ending soonest' (1,224), 'newly listed' (1,402), 'nearest first' (1,402), 'Price + Shipping: highest first' (1,403), then 'Price + Shipping: lowest first' again and I get 177 again.
Clearly there is a major problem when sorting by 'Price + Shipping: lowest first'.  Is this happening for anyone else?  Is there a solution?  How to report this to eBay support?

[eburtonlab · Accepted solution]

When you use the "Lowest First" sort, eBay will often provide a message at the top of the results:

Unless you use the "View all results" link, you are only seeing a subset of the results that eBay deems relevant for that particular sort. Other sorts do not behave that way, only Lowest First.

I get slightly different numbers of results than you do (possibly due to my shipping address or new listings being indexed or sold), but when I use that "View all results" link for that search, the number of results shown goes from 318 to 1405.

[rfmtm]

@eburtonlab  Clicking "View All Results" does not always resolve this problem.  There is a specific bug of some kind that items completely disappear when sorting on lowest price, while those items will show for any other filter selected.  This appears to be a rare problem and often goes unnoticed.  So far no fix or workaround has been found that I know of.

[eburtonlab]

There is a specific bug of some kind that items completely disappear when sorting on lowest price, while those items will show for any other filter selected. This appears to be a rare problem and often goes unnoticed.

I certainly cannot rule out a rare problem -- particularly if it only occurs intermittently. But just now when I changed the sort between Lowest First (using "view all") and Newly listed, I got 1401 results spanning eight pages at 200 items per page in both sorts.

The "View all results" link seems to only appear when there are many search results in the Lowest First sort. There may be a threshold value that triggers the relevancy test; if the number of results is below one hundred or so, the message with the link may not appear. Or there may be some other factor that accounts for its appearance.

Using the "view all" link has the effect of adding the following parameter to the page URL:

&_blrs=recall_filtering

If you encounter a situation where a Lowest First sort appears to be showing fewer results, but the "View all results" link does not appear, I would be curious to know if manually adding that parameter to the page URL restores the missing results. Perhaps the issue is that the message banner should appear at the top of the results with the link, but is missing for some reason, and the filtering is still occurring.

[rfmtm]

@eburtonlab  It is not intermittent, it just only seems to affect certain listings.  Below is an example that has been consistent for a few months.

Search for Dymo 1738595

Best filter - search indicates 40 items, but 11 are not showing

Highest, Distance, or Time filter - search will indicate 47 items and all 47 show up

Now filter by Lowest - search will still indicate 47 items, but only 36 will show.

Note that the above results happens when you search the Title, UPC, MPN, or something in the description.

Interestingly the items that go missing are all similar and from the same seller.  In this case 11 of 12 go missing, but one still shows.

Other items from other sellers have this same problem and a few have been on the boards when they finally noticed it.  It is not all the sellers items, just a few, which I assume have something in common, but different from other items from the same seller that are not affected.

I have had sellers end listings, use sell similar, reduce the words in the title, create a new listing, etc., but similar items have the same result.  What is interesting is somehow this problem interacts when the "fewer words" expanded search kicks in, and with other filter options in the left margin.

For instance.  Do the search and filter lowest.  Now enable the Free Shipping filter.  Those missing items will show up under the "fewer words" section, which makes no sense at all.

[rfmtm]

@eburtonlabWell adding that parameter you posted, makes my example search work correctly and all items show for the lowest first filter.  Unfortunately who would know to do this?  It also doesn't explain why the problem only affects certain listings and not others.

[rfmtm]

@eburtonlabJust another thought, while adding you parameter worked for my example, why doesn't clicking on View All work the same when it is available to click on? 

Wonder if there is any hope of getting ebay to look into this?

[eburtonlab]

eBay has not explained how the "relevant" results are calculated, so I can only speculate.

I seem to recall a case where a particular item appeared high up in a "Best match" sort, but did not appear in a "Lowest first" sort by default, so the criteria for each case appears to be somewhat distinct. Not that the "Best match" criteria are well-understood, either.

Perhaps the scoring relies on seller ranking to an extent -- that might explain why a certain seller's items do not appear consistently by default.

Perhaps the scoring relies on characteristics of the particular listing. Certain keywords in the title are correlated with increased sales in a category (new, guaranteed, complete, limited-edition, gold) and other keywords are correlated with decreased sales or lower prices (parts, as-is, broken, plans, incomplete, gold-plated). If an item contains enough positive terms or lacks negative terms, perhaps this contributes to a relevancy score.

eBay's search makes a lot of assumptions about what a user is actually looking for, and attempts to show items that will lead to sales. For the average, marginal user, the default assumptions may work well enough; for power-users looking for literal and logically-complete results, taking steps to bypass eBay's default assumptions may be necessary. Sellers, of course, will want to try to make sure their items appear in the relevant search results where potential customers will be steered by eBay's "smart" search.

[eburtonlab]

Search for Dymo 1738595

Best filter - search indicates 40 items, but 11 are not showing

Highest, Distance, or Time filter - search will indicate 47 items and all 47 show up

Now filter by Lowest - search will still indicate 47 items, but only 36 will show.

Interesting. I get similar results. I note that adding the &_blrs=recall_filtering to the Best Match search URL has no effect on the number of listings shown; apparently whatever is causing the "Best Match" results to be lower is not the same sort of filtering as the "Lowest First" sort.

[eburtonlab]

Just another thought, while adding you parameter worked for my example, why doesn't clicking on View All work the same when it is available to click on?

Clicking on "View all results" does work that way for me -- when I use that link I see the missing results for the Lowest Price sort.

[rfmtm]

Generally the View All works for me as well, but not always depending on what other filters are enabled at the same time as things strangely more to the Fewer Words section.   Takes a bit of time to do all the variations, but it is clear something is wrong.  And of course in my example the View All button is not even available to click on, or it probably would work since adding your posted parameter to the URL works.

[eburtonlab]

Generally the View All works for me as well, but not always depending on what other filters are enabled at the same time as things strangely more [move?] to the Fewer Words section.

If listings that match all your search terms are appearing in the "matching fewer words" section, that might be a sign that eBay is doing some unwanted keyword grouping/keyword substitution.

Try using your keywords in a different order to break up the keyword grouping. Reversing the order of the keywords often works well.

There are other possible workarounds such as using an exclusion term or putting one of your keywords in quotes, but these have other side effects: using such search commands will disable the search expansion features including any keyword substitution or automatic keyword to category navigation. Which may be fine if you just want to find literal matches of your keywords, but you may lose some results that are slight variations. Sometimes multiple searches are required to cover all the possibilities.

[rfmtm]

Yes, doing that does help sometimes, but often it just generates other unwanted results.  Note also that this problem exists even when you search on a single unique term only relevant to a specific item.  

For example, search only on this UPC number:  611885671911

Without the View All parameter, the Low filter search indicates 12 items, but only one item shows up and the rest are missing from view.   Any other filter shows all 12.

I often search by UPC, or MPN, or something unique to cut down on the irrelevant results, as I am sure others do.  But I am sure few realize not all items are viewable, if no View All option appears to click on.

[gazarcher]

Apologies for the late reply...

I did my search again and this time I did see the "view all results" link and I saw better results.  The difference was significant though.

(i) Before I selected the 'lowest' filter (searching for 'Best Match') there were 1,261 results.

(ii) After selecting the lowest price filter, there are now 152 results.

(iii) Then I select 'View all results' and now there are 1,293 results -- 32 more than before!

(iv) I did the 'Best Match' search again to verify the results and I got 1,261 results again.

Clearly the 'lowest' filter search is either more efficient -- or incorrect -- than the 'best' filter search or there is a bug with whichever sort filter you select.

I have been following the interesting discussion -- thank you for all your help, discoveries and suggestions -- and it is clear to me that eBay's searching algorithms leave me with zero confidence that they will ever give you a correct result.

To have one filter give you 'x' results and another filter give you, for example, x+y or x-z results and finding the results to not be identical is not acceptable to any user.  You don't know which one of those three results is correct!  It thus requires either accepting whatever results you get or continuously experiment with the search criteria to get a more consistent result.

I am a retired senior software engineer who has worked with sorting and searching algorithms over decades of my career since when I was a junior programmer just starting out (I have been a 'C', C++ and Java expert) and I know it can be a difficult task to achieve the consistent results you expect when dealing with huge datasets that are constantly changing, like eBay's.  EBay certainly has absolutely *HUGE* datasets of auctions, current and previous, never mind purchase histories, messages, etc!  No doubt eBay has an army of software engineers and database experts and I suspect there are many whose full-time job it is, is just to chase sorting and searching bugs!  I am not jealous of working on that particular task!  (Been there, done that, chasing bugs.)

In any case, I deeply appreciate those that have spent time to answer my original question and I have been impressed with your knowledge of the sorting and searching dilemmas and how to try to resolve them.  THANK YOU SO MUCH!

[rfmtm]

Back in the beginning, ebay did a simple boolean search, that worked great with predicable and consistent results.  The first sign of trouble was when Best Search came along, and soon after that full flat out search manipulation with further complications from ebay catalog errors.

Now, even if you think you have figured out how to search or list, another change or glitch is just around the corner.  Worse is everyone is going this route including Amazon and even search engines like Google.  They just ignore specific parameters in your search and show you what they want you to see.