Provide means of configuring fields to sort on
Description
is followed up by
Activity
Show:
Gunnar Morling September 8, 2015 at 9:39 AM
1# if the field is in a _ToMany relation (or iterated in any other way) it's not valid, although it might be valid for other target types if they are indexed as well
The current PR will not add a doc value field in the case a sort field is reached through a *-to-many association.
Sanne Grinovero September 7, 2015 at 5:30 PM
A first proposal:
https://gist.github.com/gunnarmorling/4ab262cf1fc772047897
And some further considerations happened on IRC:
[16:53] <sannegrinovero> emmanuel: looking at the @SortField discussion, there are two aspects which need to be further analysed
[16:54] <sannegrinovero> 1# if the field is in a _ToMany relation (or iterated in any other way) it's not valid, although it might be valid for other target types if they are indexed as well
[16:55] <sannegrinovero> 2# how to link it to index fields which are not directly generated by the annotated property?
[17:00] <emmanuel> how do you put a @SortedField on a ToMany? Via @IndexedEmbedded sannegrinovero ?
[17:00] <emmanuel> if yes, then that's not worse than before
[17:00] <emmanuel> the annotation would be valid in some cases but not in the embedded case
[17:01] <sannegrinovero> emmanuel: right, but what do you do in terms of validation?
[17:01] <emmanuel> hippie style
[17:01] <emmanuel> let go
[17:01] <sannegrinovero> I guess we can log a warning at runtime when the fieldname is chosen for a sorted query
[17:01] <sannegrinovero> but you can't throw up at boot time
[17:02] <emmanuel> yes, the query time seems like a good slot to warn against that
[17:02] <emmanuel> or even through an exception
[17:02] <sannegrinovero> right, this case could even be an exception.
[17:02] <emmanuel> can you elaborate on 2?
[17:02] <emmanuel> custom field bridge for example?
[17:02] <emmanuel> or are you thinking of someting else?
[17:03] <sannegrinovero> yea, a custom fieldbridge or even a ClassBridge
[17:03] <sannegrinovero> A custom FieldBridge could still somewhat be related to an entity property, but the ClassBridge doesn't have that link
[17:03] <emmanuel> if that's a classbridge, I think a class level @SortField-ish annotation could be used
[17:04] <emmanuel> I dont' know enough of how that's implemented to know the real link between a property and the doc value generation
[17:04] <emmanuel> so I don't have a grasp on how custom field bridges play here
[17:04] <sannegrinovero> Ok, so shoul we allow the type on class already, or should we not make early concessions..
[17:05] <sannegrinovero> I'd be tempted to start this w/o an option for fields defined by a ClassBridge, just pointing out the limitation
[17:05] <emmanuel> i thnk that would be my current preference
[17:05] <emmanuel> I'd rather we force bridges ot provide field metadata
[17:06] <sannegrinovero> Me too. But does it affect your overall view on how this metadata should be defined:
[17:06] <sannegrinovero> it really looks like the "sort fields" are strongly coupled to the root entity only, and not so much with the properties
[17:07] <sannegrinovero> So for example - thinking out of the box - it could be just a list of field names defined by an annotation on the root type.
[17:07] <sannegrinovero> (that's considering limitation #1 as well)
[17:09] <emmanuel> sannegrinovero: so would be fields I guess
[17:09] <emmanuel> ah no because the sort field would feed (data wise) from the regular field
[17:09] <emmanuel> so why is @Field(sort=YES) insufficient?
[17:10] <emmanuel> in which case is sorta solves the bridge issue as well as the bridge would drive sortability like it drives indexability
[17:11] <sannegrinovero> generally a @Field is bound to an entity property to have it "generate fields", but in the sorting case you want to know which one of the generated fields..
[17:12] <sannegrinovero> I think there's a mismatch with @Field as that annotation is treated recursively via @IndexedEmbedded, while in this case we don't want that kind of recursion
[17:13] <sannegrinovero> emmanuel: different but related question: should we allow to sort on a *property name*?
[17:13] <emmanuel> what did we allow before?
[17:13] <emmanuel> I am not sure why would would or even could revisit that in a 5.5
[17:13] <sannegrinovero> we allowed anything, and no validations.
[17:14] <sannegrinovero> althought since the user was not being helped (no sorting DSL), the user had to play with document field names (definitely not property names)
[17:14] <emmanuel> we still don't ahve a sort DSL right?
[17:15] <sannegrinovero> right
[17:15] <emmanuel> in a ideal world, I would want people to sort by property
[17:15] <emmanuel> searhc by property
[17:15] <emmanuel> everything at the object level
[17:15] <emmanuel> de factor
[17:15] <emmanuel> o
[17:15] <sannegrinovero> I'm asking this now - even if we don't want to do this now - because it would be good to know if we should inherently re-map (one day) a property whic his mapped to multiple fields
[17:15] <sannegrinovero> into a sort on multiple fields
[17:15] <emmanuel> but with option to shift gear towards plain Lucene level as we have seen that i is useful for powerusers
[17:16] <sannegrinovero> Ok, yes that's what I wanted to check. Thanks.
[17:17] <sannegrinovero> so when we revisit FieldBridge SPI & co we should not only make sure to track which fields it's going to generate but also to have it generate multi-field sort definitions
[17:19] <emmanuel> sannegrinovero: right, for example, a year/month/day multi field bridge would know how to sort
[17:19] <emmanuel> a price + currency *might* know
[17:19] <emmanuel> a Map<k,v> to map.k = v will not know
[17:20] <emmanuel> a bridge storing in different language is screwed too :D
[17:20] <emmanuel> this is tough
[17:21] <sannegrinovero> I guess we'll have to treat is a an "aspect"
[17:21] <sannegrinovero> not mandatory for all bridges
[17:23] <emmanuel> I think I would like this aspect o be about "sortability"
[17:23] <emmanuel> and the user have the abiloty to enable sorting or not on the field using that bridge or not
[17:23] <emmanuel> that's the most inuitive to me
[17:23] <emmanuel> and indeed we might need to do like @IndexedEmbedded and offer a flat list like you say
[17:24] <emmanuel> for sorted properties form a root
[17:27] <sannegrinovero> emmanuel: so you'd expect a set of "SortabilityBridge" implementations, one per type, as a parallel concept to Fieldbridge and we allow some form of mix&match with different bridges and sorters?
[17:28] <sannegrinovero> It all needs to be agnostic of Lucene types too :)
[17:28] <emmanuel> why would you separate it from FieldBridge?
[17:29] <emmanuel> to me a sortable thing would be a fieldbridge
[17:29] <emmanuel> or classbridge
[17:30] <sannegrinovero> ok, that's what I was thinking too when I said an "aspect", but then you seemed to imply you'd make it more of a separate thing
[17:30] <emmanuel> I don't think there is much benefits into spliting it
[17:31] <emmanuel> what I mean rather is that a sorrableFieldBridge might not generate a doc value if the user has not set @SortBridge (or whatever that is)
[17:31] <sannegrinovero> BTW at the end of the day we should really just provide the "flat list" of everything, not just sort fields, but engage the @IndexedEmbedded recursion capabilities just as a convenience to have the users make their shopping list at least effort.
[17:31] <sannegrinovero> ok, looks we agree then.
With Lucene 5 we should provide a way for explicitly configuring sort fields, avoiding the need for un-inverting when reading.