Object fields that are both nested and flattened (ES's include_in_parent)

Description

While nested documents provide additional features, we don't needed nested documents for everything. Predicates on a single field when we're not using the nested predicate, in particular, work perfectly well on flattened object fields, and actually perform better since we don't need a runtime join.

To get the best of both worlds, when one can spare the extra disk space it requires, we could allow object fields to be stored both as a nested document and flattened in their parent. Elasticsearch allows that with the attribute include_in_parent (see https://www.elastic.co/guide/en/elasticsearch/reference/current/nested.html#nested-params).

API-wise, this could just be an additional value for the ObjectStructure enum: ObjectStructure.NESTED_AND_FLATTENED. We could then take advantage of this internally: when nesting is not necessary (no nested predicate, no filter in sorts and aggregations), we can just use the flattened field.

Activity

Details

Assignee

Reporter

Components

Priority

Created June 19, 2020 at 6:38 AM
Updated January 8, 2025 at 1:04 PM