Defining Domain Specific Blocking And Similarity Functions
To add blocking functions and how they work
The blocking tree works on the matched records provided by the user as part of the training. At every node, it selects the hash function and the field on which it should be applied so that there is the least elimination of the matching pairs. Say we have data like this:
Let us assume we have hash function first1char and we want to check if it is a good function to apply to firstname:
There is no elimination in the pairs above, hence it is a good function.
Now let us try last1char on firstname
Pair 1 is getting eliminated above, hence last1char is not a good function.
So, first1char(firstname) will be chosen. This brings near similar records together - in a way, clusters them to break the cartesian join.