Merge duplicates (advanced)

Merge records of a duplicate group, as identified using the Duplicate Detection analyzer.

Each output record of this transformation will get a 'Merge status' field which can have either of these values:

  • SURVIVOR - if the record is to be preserved as the survivor of the merge operation.
  • NON_SURVIVOR - if the record is to be discarded/deleted/invalidated as the result of the merge operation.
  • UNIQUE - if the record was not a duplicate (group size was 1 or group ID was null).
  • MANUAL - if the a rule set in the configuration of this transformation caused the duplicate group to be a candidate for manual merging.
  • FAILURE - if a failure occurred while merging.
Transformer Concurrent Alias: EasyDQ Merge duplicates
  • Values

    Values that will enter the merge configuration

    List of InputColumn<Object> Required
  • Group ID

    Column containing a unique group ID

    InputColumn<Object> Required
  • Group size

    Column containing the amount of records in the group

    InputColumn<Object> Required
  • Configuration

    ClientMergeConfiguration Required
  • Output non survivor records

    Generate empty records for each non-merged/non-survivor record?

    boolean Required
  • Threshold column

    Column containing a manual merge threshold value.

    InputColumn<Number> Optional
  • Threshold value

    Threshold values for manual merging. In case actual threshold values are below this value, the records will be marked as manual merge cases.

    Double Optional