Contextual filter for Search API not working

I am working on a Drupal 7.53 site and using the Search API module (7.x-1.18). I have an index of two content types, both of which use the same taxonomy tags. In my View I’ve created a Contextual Filter using “Search: Indexed taxonomy term field”. I have used this view on other indices with a single content type and it works fine.

Inside the filter I have checked “Provide default argument” and “Taxonomy term ID from URL”. I also have the “Load default filter from node page” checked and am not limiting the vocabularies to any particular one. Finally I have selected “Filter to items that share any term”.

The problem is that is sucks in all the content of the entire index and does not filter any way. What could I be doing wrong here? Is it that the index is not set up properly? The view? Or something else?

Drupal version: