== Filtering Queries and Aggregations

A natural extension to aggregation scoping is filtering. Because the aggregation operates in the context of the query scope, any filter applied to the query will also apply to the aggregation.

[float="true"] === Filtered Query If we want to find all cars over $10,000 and also calculate the average price for those cars,((("filtering", "serch query results")))((("filtered query")))((("queries", "filtered"))) we can simply use a filtered query:

[source,js]

GET /cars/transactions/_search?search_type=count { "query" : { "filtered": { "filter": { "range": { "price": { "gte": 10000 } } } } }, "aggs" : { "single_avg_price": { "avg" : { "field" : "price" } } }

}

// SENSE: 300_Aggregations/45_filtering.json

Fundamentally, using a filtered query is no different from using a match query, as we discussed in the previous chapter. The query (which happens to include a filter) returns a certain subset of documents, and the aggregation operates on those documents.

[float="true"] === Filter Bucket

But what if you would like to filter just the aggregation results?((("filtering", "aggregation results, not the query")))((("aggregations", "filtering just aggregations"))) Imagine we are building the search page for our car dealership. We want to display search results according to what the user searches for. But we also want to enrich the page by including the average price of cars (matching the search) that were sold in the last month.

We can't use simple scoping here, since there are two different criteria. The search results must match +ford+, but the aggregation results must match +ford+ AND +sold > now - 1M+.

To solve this problem, we can use a special bucket called filter.((("filter bucket")))((("buckets", "filter"))) You specify a filter, and when documents match the filter's criteria, they are added to the bucket.

Here is the resulting query:

[source,js]

GET /cars/transactions/_search?search_type=count { "query":{ "match": { "make": "ford" } }, "aggs":{ "recent_sales": { "filter": { <1> "range": { "sold": { "from": "now-1M" } } }, "aggs": { "average_price":{ "avg": { "field": "price" <2> } } } } }

}

// SENSE: 300_Aggregations/45_filtering.json

<1> Using the filter bucket to apply a filter in addition to the query scope.

<2> This avg metric will therefore average only docs that are both +ford+ and sold in the last month.

Since the filter bucket operates like any other bucket, you are free to nest other buckets and metrics inside. All nested components will "inherit" the filter. This allows you to filter selective portions of the aggregation as required.

[float="true"] === Post Filter

So far, we have a way to filter both the search results and aggregations (a filtered query), as well as filtering individual portions of the aggregation (filter bucket).

You may be thinking to yourself, "hmm...is there a way to filter just the search results but not the aggregation?"((("filtering", "search results, not the aggregation")))((("post filter"))) The answer is to use a post_filter.

This is a top-level search-request element that accepts a filter. The filter is applied after the query has executed (hence the +post+ moniker: it runs post query execution). Because it operates after the query has executed, it does not affect the query scope--and thus does not affect the aggregations either.

We can use this behavior to apply additional filters to our search criteria that don't affect things like categorical facets in your UI. Let's design another search page for our car dealer. This page will allow the user to search for a car and filter by color. Color choices are populated via an aggregation:

[source,js]

GET /cars/transactions/_search?search_type=count { "query": { "match": { "make": "ford" } }, "post_filter": { <1> "term" : { "color" : "green" } }, "aggs" : { "all_colors": { "terms" : { "field" : "color" } } }

}

// SENSE: 300_Aggregations/45_filtering.json

<1> The post_filter element is a +top-level+ element and filters just the search hits.

The query portion is finding all +ford+ cars. We are then building a list of colors with a terms aggregation. Because aggregations operate in the query scope, the list of colors will correspond with the colors that Ford cars are painted.

Finally, the post_filter will filter the search results to show only green +ford+ cars. This happens after the query is executed, so the aggregations are unaffected.

This is often important for coherent UIs. Imagine that a user clicks a category in your UI (for example, green). The expectation is that the search results are filtered, but not the UI options. If you applied a filtered query, the UI would instantly transform to show only +green+ as an option--not what the user wants!

[WARNING]

.Performance consideration

Use a post_filter only if you need to differentially filter search results and aggregations. ((("post filter", "performance and")))Sometimes people will use post_filter for regular searches.

Don't do this! The nature of the post_filter means it runs after the query, so any performance benefit of filtering (such as caches) is lost completely.

The post_filter should be used only in combination with aggregations, and only

when you need differential filtering.

[float="true"] === Recap

Choosing the appropriate type of filtering--search hits, aggregations, or both--often boils down to how you want your user interface to behave. Choose the appropriate filter (or combinations) depending on how you want to display results to your user.

• A filtered query affects both search results and aggregations.
• A filter bucket affects just aggregations.
• A post_filter affects just search results.