Uploaded image for project: 'Hibernate Search'
  1. HSEARCH-2458

Report incomplete metadata issues during Elasticsearch mapping generation

    Details

    • Type: Improvement
    • Status: Closed
    • Priority: Minor
    • Resolution: Fixed
    • Affects Version/s: 5.6.0.Beta3
    • Fix Version/s: 5.8.0.Beta2
    • Component/s: backend-elasticsearch
    • Labels:
      None

      Description

      Issue spotted here: https://github.com/hibernate/hibernate-search/pull/1217/commits/dfc7f199358808397d144d79d89f7915851db222#r87074164

      Right now, we're only logging debug messages when failing to add a property/field to the ES mapping:

      		// normal document fields
      		for ( DocumentFieldMetadata fieldMetadata : typeMetadata.getNonEmbeddedDocumentFieldMetadata() ) {
      			try {
      				addPropertyMapping( mappingBuilder, fieldMetadata );
      			}
      			catch (IncompleteDataException e) {
      				LOG.debug( "Not adding a mapping for field " + fieldMetadata.getAbsoluteName() + " because of incomplete data", e );
      			}
      		}
      
      		// bridge-defined fields
      		for ( BridgeDefinedField bridgeDefinedField : getNonEmbeddedBridgeDefinedFields( typeMetadata ) ) {
      			try {
      				addPropertyMapping( mappingBuilder, bridgeDefinedField );
      			}
      			catch (IncompleteDataException e) {
      				LOG.debug( "Not adding a mapping for field " + bridgeDefinedField.getAbsoluteName() + " because of incomplete data", e );
      			}
      		}
      
      ....
      
      
      		for ( FacetMetadata facetMetadata : fieldMetadata.getFacetMetadata() ) {
      			try {
      				addFieldMapping( propertyMapping, mappingBuilder, facetMetadata );
      			}
      			catch (IncompleteDataException e) {
      				LOG.debug( "Not adding a mapping for facet " + facetMetadata.getAbsoluteName() + " because of incomplete data", e );
      			}
      		}
      

      The only way to get IncompleteDataException can be found in addTypeOptions:

      ...
      				break;
      			case UNKNOWN_NUMERIC:
      				// Likely a custom field bridge which does not expose the type of the given field; either correctly
      				// so (because the given name is the default field and this bridge does not wish to use that field
      				// name as is) or incorrectly; The field will not be added to the mapping, causing an exception at
      				// runtime if the bridge writes that field nevertheless
      				elasticsearchType = null;
      				break;
      			case STRING:
      			case UNKNOWN:
      			default:
      				elasticsearchType = DataType.STRING;
      				break;
      		}
      
      		if ( elasticsearchType == null ) {
      			throw new IncompleteDataException( "Field type could not be determined" );
      		}
      

      While I understand that we might not want to make the mapping generation fail completely (so that users may test more easily), at least we should issue a warning. Or maybe even log an error.

      Something to be considered: Sanne Grinovero expressed concerned about issuing warnings, since in some enterprises they are considered blocking when putting applications in production.

      I feel like one of the purposes of warnings is to inform users about potential error, and leave it to the user to decide if it's bad or not. So in this case, it would be exactly what we need. Now, if it's blocking for some users...

      Guillaume Smet, Sanne Grinovero, WDYT?

        Attachments

          Activity

            People

            • Assignee:
              yrodiere Yoann Rodière
              Reporter:
              yrodiere Yoann Rodière
            • Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: