> For the complete documentation index, see [llms.txt](https://docs.zingg.ai/zingg/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://docs.zingg.ai/zingg/running-zingg/configure-zingg.md).

# Configure Zingg

Set up your field definitions, pipes, and arguments - the foundation every subsequent Zingg phase depends on.

Configuring Zingg is the first step in every Zingg workflow. This page follows notebook 01 step by step: defining your fields, configuring your pipes, and creating the arguments object that all subsequent phases inherit.

Complete this page before running any other phase. Every phase reads the configuration set here:

* `findTrainingData` → \[[Create Training Data](/zingg/running-zingg/create-training-data.md)]
* `label` → \[[Label Training Pairs](/zingg/running-zingg/create-training-data/label-training-pairs.md)]
* `train` → \[[Build and Save the Model](/zingg/running-zingg/build-and-save-the-model.md)]
* `match` → \[[Run the Match Phase](/zingg/running-zingg/run-the-match-phase.md)]
* `link` → \[[Link Across Datasets](/zingg/running-zingg/link-across-datasets.md)]
* `runIncremental` → \[[Run Incremental Matching](/zingg/running-zingg/run-incremental-matching.md)]

{% hint style="success" icon="right-long" %}
Full parameter reference with all config options and schema → [Configuration Schema](/zingg/reference/configuration-schema.md)
{% endhint %}

{% tabs %}
{% tab title="Community" %}

### Step 1: Verify installation

```python
from zingg.client import *
from zingg.pipes import *
```

Verify in your notebook

```python
!pip show zingg
```

### Step 2: Build the arguments object

#### Python

```python
args = Arguments()
args.setModelId("100")
args.setZinggDir("models")
args.setNumPartitions(4)
args.setLabelDataSampleSize(0.5)
```

{% hint style="success" icon="right-long" %}
`setModelId` - unique name for this model.

`setZinggDir` - where Zingg writes model files and training data.

Use the same `modelId` in all subsequent phases for this run.
{% endhint %}

#### JSON

```json
{
  "modelId" : "100",
  "zinggDir" : "models",
  "numPartitions" : 4,
  "labelDataSampleSize" : 0.5
}
```

#### Understanding `numPartitions` and `labelDataSampleSize`

**`numPartitions`** - The number of Spark partitions over which input data is distributed. Set this to approximately 20–30 times the number of worker cores. This is the most important configuration for performance.

**`labelDataSampleSize`** - Fraction of the data scanned to find candidate pairs. Adjust between `0.0001` and `0.1`. If `findTrainingData` is slow, reduce to `0.05` or lower. If it is too small, Zingg may not find the right edge cases.

### Step 3: Define fields and match types

#### Python

```python
id = FieldDefinition("id", "string", MatchType.DONT_USE)
fname = FieldDefinition("fname", "string", MatchType.FUZZY)
lname = FieldDefinition("lname", "string", MatchType.FUZZY)
stNo = FieldDefinition("stNo", "string", MatchType.FUZZY)
add1 = FieldDefinition("add1", "string", MatchType.FUZZY)
add2 = FieldDefinition("add2", "string", MatchType.FUZZY)
city = FieldDefinition("city", "string", MatchType.FUZZY)
areacode = FieldDefinition("areacode", "string", MatchType.FUZZY)
state = FieldDefinition("state", "string", MatchType.FUZZY)
dob = FieldDefinition("dob", "string", MatchType.FUZZY)
ssn = FieldDefinition("ssn", "string", MatchType.FUZZY)

fieldDefs = [
  id, fname, lname, stNo, add1, add2, city,
  areacode, state, dob, ssn
]
args.setFieldDefinition(fieldDefs)
```

#### JSON

```json
{
  "fieldDefinition" : [
    {
      "fieldName" : "id",
      "matchType" : "DONT_USE",
      "fields" : "id",
      "dataType" : "string"
    },
    {
      "fieldName" : "fname",
      "matchType" : "FUZZY",
      "fields" : "fname",
      "dataType" : "string"
    },
    {
      "fieldName" : "lname",
      "matchType" : "FUZZY",
      "fields" : "lname",
      "dataType" : "string"
    },
    {
      "fieldName" : "stNo",
      "matchType" : "FUZZY",
      "fields" : "stNo",
      "dataType" : "string"
    },
    {
      "fieldName" : "add1",
      "matchType" : "FUZZY",
      "fields" : "add1",
      "dataType" : "string"
    },
    {
      "fieldName" : "add2",
      "matchType" : "FUZZY",
      "fields" : "add2",
      "dataType" : "string"
    },
    {
      "fieldName" : "city",
      "matchType" : "FUZZY",
      "fields" : "city",
      "dataType" : "string"
    },
    {
      "fieldName" : "areacode",
      "matchType" : "FUZZY",
      "fields" : "areacode",
      "dataType" : "string"
    },
    {
      "fieldName" : "state",
      "matchType" : "FUZZY",
      "fields" : "state",
      "dataType" : "string"
    },
    {
      "fieldName" : "dob",
      "matchType" : "FUZZY",
      "fields" : "dob",
      "dataType" : "string"
    },
    {
      "fieldName" : "ssn",
      "matchType" : "FUZZY",
      "fields" : "ssn",
      "dataType" : "string"
    }
  ]
}
```

{% hint style="success" icon="right-long" %}
**Read more**: Match types reference - [Match types](/zingg/zingg-concepts/zingg-configuration/field-definition/match-types.md) | [Configuration schema](/zingg/reference/configuration-schema.md)
{% endhint %}

### Step 4: Configure input and output pipes

#### Python

```python
schema = "id string, fname string, \
lname string, stNo string, add1 string, \
add2 string, city string, areacode string,\
 state string, dob string, ssn string"

inputPipe = CsvPipe("testFebrl", "examples/febrl/test.csv", schema)
args.setData(inputPipe)

outputPipe = CsvPipe("resultFebrl", "/tmp/febrlOutput")
args.setOutput(outputPipe)
```

#### JSON

```json
{
  "data" : [ {
    "name" : "testFebrl",
    "format" : "csv",
    "props" : {
      "path" : "examples/febrl/test.csv",
      "delimiter" : ",",
      "header" : "true"
    },
    "schema" : "id string, fname string, lname string, stNo string, add1 string, add2 string, city string, areacode string, state string, dob string, ssn string"
  } ],
  "output" : [ {
    "name" : "resultFebrl",
    "format" : "csv",
    "props" : {
      "path" : "/tmp/febrlOutput",
      "delimiter" : ",",
      "header" : "true"
    }
  } ]
}
```

{% hint style="success" icon="right-long" %}
**Read more**: For all supported connector types and formats - [Connect Your Data](/zingg/connect-your-data/pipes-and-data-connections.md)
{% endhint %}
{% endtab %}

{% tab title="Enterprise" %}

### Step 1: Verify installation and imports

```python
from zingg.client import *
from zingg.pipes import *
from zinggEC.enterprise.common.epipes import *
from zinggEC.enterprise.common.EArguments import *
from zinggEC.enterprise.common.EFieldDefinition import EFieldDefinition
```

Verify the `zinggEC` package is installed by running `!pip show zinggEC` in a separate cell.

### Step 2: Build the Enterprise arguments object

```python
args = EArguments()
args.setModelId("100")
args.setZinggDir("/tmp/models")
args.setNumPartitions(4)
args.setLabelDataSampleSize(0.5)
```

Set the blocking strategy if needed. If not set, the model follows `DEFAULT`. See [Blocking Strategies: DEFAULT vs WIDER](/zingg/tuning/blocking-strategy.md) for how each strategy orders fields.

#### Python

```python
args.setBlockingModel("DEFAULT")
```

#### JSON

```json
{
  "modelId": "100",
  "zinggDir": "/tmp/models",
  "numPartitions": 4,
  "labelDataSampleSize": 0.5,
  "blockingModel": "DEFAULT"
}
```

### Step 3: Define fields with `EFieldDefinition`

#### Python

```python
recId = EFieldDefinition(
    "recId",
    "string",
    MatchType.DONT_USE
)
recId.setPrimaryKey(True)

fname = EFieldDefinition(
    "fname",
    "string",
    MatchType.FUZZY
)
lname = EFieldDefinition(
    "lname",
    "string",
    MatchType.FUZZY
)
stNo = EFieldDefinition(
    "stNo",
    "string",
    MatchType.FUZZY
)
add1 = EFieldDefinition(
    "add1",
    "string",
    MatchType.FUZZY
)
add2 = EFieldDefinition(
    "add2",
    "string",
    MatchType.FUZZY
)
city = EFieldDefinition(
    "city",
    "string",
    MatchType.FUZZY
)
areacode = EFieldDefinition(
    "areacode",
    "string",
    MatchType.FUZZY
)
state = EFieldDefinition(
    "state",
    "string",
    MatchType.FUZZY
)
dob = EFieldDefinition(
    "dob",
    "string",
    MatchType.FUZZY
)
ssn = EFieldDefinition(
    "ssn",
    "string",
    MatchType.FUZZY
)

fieldDefs = [
    recId,
    fname,
    lname,
    stNo,
    add1,
    add2,
    city,
    areacode,
    state,
    dob,
    ssn
]
args.setFieldDefinition(fieldDefs)
```

{% hint style="info" icon="right-long" %}
Enterprise requires a primary key field for `runIncremental`. Mark the primary key field by calling `recId.setPrimaryKey(True)` if you plan to use incremental matching.
{% endhint %}

### Step 4: Configure input and output pipes

#### Python

```python
schema = ("recId string, fname string, "
          "lname string, stNo string, "
          "add1 string, add2 string, "
          "city string, areacode string, "
          "state string, dob string, "
          "ssn string")

inputPipe = ECsvPipe(
    "testFebrl",
    "examples/febrl/test.csv",
    schema
)
args.setData(inputPipe)

outputPipe = ECsvPipe(
    "resultFebrl",
    "/tmp/febrlOutput"
)
outputPipe.setHeader("true")
args.setOutput(outputPipe)
```

#### JSON

```json
{
  "data": [ {
    "name": "testFebrl",
    "format": "csv",
    "props": {
      "path": "examples/febrl/test.csv",
      "delimiter": ",",
      "header": "true"
    },
    "schema": "recId string, fname string, lname string, stNo string, add1 string, add2 string, city string, areacode string, state string, dob string, ssn string"
  } ],
  "output": [ {
    "name": "resultFebrl",
    "format": "csv",
    "props": {
      "path": "/tmp/febrlOutput",
      "delimiter": ",",
      "header": "true"
    }
  } ]
}
```

### Step 5: Deterministic matching (optional)

{% hint style="info" icon="right-long" %}
Deterministic matching - **Enterprise** only.

Skip this step if you only need probabilistic matching.
{% endhint %}

#### Python

```python
detMatchNameAdd = DeterministicMatching(
    'fname',
    'stNo',
    'add1'
)
detMatchNameDobSsn = DeterministicMatching(
    'fname',
    'dob',
    'ssn'
)
detMatchLnameDob = DeterministicMatching(
    'lname',
    'dob'
)
args.setDeterministicMatchingCondition(
    detMatchNameAdd,
    detMatchNameDobSsn,
    detMatchLnameDob
)
```

#### JSON

```json
{
  "deterministicMatching": [
    {"matchCondition": [
      {"fieldName": "fname"},
      {"fieldName": "stNo"},
      {"fieldName": "add1"}
    ]},
    {"matchCondition": [
      {"fieldName": "fname"},
      {"fieldName": "dob"},
      {"fieldName": "ssn"}
    ]},
    {"matchCondition": [
      {"fieldName": "lname"},
      {"fieldName": "dob"}
    ]}
  ]
}
```

### Step 6: Output stats (optional)

{% hint style="info" icon="right-long" %}
Output stats - **Enterprise** only.

Skip this step if you do not need run statistics.
{% endhint %}

Configures where Zingg writes statistics for the `match` and `runIncremental` phases. The `$ZINGG_DYNAMIC_STAT_NAME` placeholder is substituted at runtime with `SUMMARY`, `CLUSTER`, or `RECORD`, so each run writes the three statistics files separately. If `outputStats` is not configured, Zingg skips stats writing and the run proceeds normally.

#### Python

```python
statsPipe = ECsvPipe(
    "stats",
    "/tmp/febrlStats_$ZINGG_DYNAMIC_STAT_NAME"
)
args.setOutputStats(statsPipe)
```

#### JSON

```json
{
  "outputStats": {
    "name": "stats",
    "format": "csv",
    "props": {
      "location": "/tmp/zinggStats_$ZINGG_DYNAMIC_STAT_NAME",
      "delimiter": ",",
      "header": true
    }
  }
}
```

**Read more:** For the fields in each statistics table → [Output Statistics](/zingg/interpreting-results/explainability-and-statistics/output-statistics.md).

### Step 7: Pass Through (optional)

{% hint style="info" icon="right-long" %}
Pass Through - **Enterprise** only.

Skip this step if you do not need to exclude records from matching.
{% endhint %}

Pass Through excludes specific records from matching while still including them in output with their own Zingg ID. Records matching the passthrough expression appear in the identity graph but never influence cluster formation.

**Note**: Zingg internally applies the negation of `passthroughExpr` to filter matching records. If the passthrough condition applies to nullable fields, ensure the negative of the expression yields the records that are `NOT` passthrough.

#### Python

```python
args.setPassthroughExpr("fname = 'matilda'")
```

#### JSON

```json
{
  "passthroughExpr": "fname = 'matilda'"
}
```

#### Example with null-safe expression

```json
{
  "passthroughExpr": "is_deceased = true AND is_deceased is NOT NULL"
}
```

{% endtab %}

{% tab title="Enterprise Snowflake" %}

### Step 1: Build the Enterprise arguments object

#### JSON

```json
{
  "modelId": "100",
  "zinggDir": "/tmp/models",
  "numPartitions": 4,
  "labelDataSampleSize": 0.5
}
```

### Step 2: Define fields and match types

#### JSON

```json
  "fieldDefinition":[
		{
			"fieldName" : "recid",
			"matchType" : "dont_use",
			"fields" : "recid",
			"dataType": "string" ,
			"primaryKey": "true"
		},
		{
			"fieldName" : "givenname",
			"matchType" : "fuzzy",
			"fields" : "givenname",
			"dataType": "string" 
		},
		{
			"fieldName" : "surname",
			"matchType": "exact",
			"fields" : "surname",
			"dataType": "string" 
		},
		{
			"fieldName" : "suburb",
			"matchType": "fuzzy",
			"fields" : "suburb",
			"dataType": "string" 
		},
		{
			"fieldName" : "postcode",
			"matchType": "exact",
			"fields" : "postcode",
			"dataType": "string" 
		}
	]
```

{% hint style="info" icon="right-long" %}
Enterprise requires a primary key field for `runIncremental`. Mark the primary key field by adding `"primaryKey": "true"` if you plan to use incremental matching.
{% endhint %}

### Step 3: Configure input and output pipes

#### JSON

```json
	"output" : [{
		"name":"output_Febrl_5M",
		"format":"snowflake",
		"props": {
			"table": "OUTPUT_Febrl_5M"
		}
	}],
	"data" : [{
		"name":"FEBRL5M",
		"format":"snowflake",
		"props": {
			"table": "Febrl_5M"
		}
	}],
```

### Step 4: Deterministic matching (optional)

{% hint style="info" icon="right-long" %}
Deterministic matching - **Enterprise** only.

Skip this step if you only need probabilistic matching.
{% endhint %}

#### JSON

```json
{
  "deterministicMatching": [
    {"matchCondition": [
      {"fieldName": "givenname"},
      {"fieldName": "surname"},
      {"fieldName": "postcode"}
    ]},
    {"matchCondition": [
      {"fieldName": "givenname"},
      {"fieldName": "surname"},
      {"fieldName": "suburb"}
    ]}
  ]
}
```

### Step 5: Output stats (optional)

{% hint style="info" icon="right-long" %}
Output stats - **Enterprise** only.

Skip this step if you do not need run statistics.
{% endhint %}

Configures where Zingg writes statistics for the `match` and `runIncremental` phases. The `$ZINGG_DYNAMIC_STAT_NAME` placeholder is substituted at runtime with `SUMMARY`, `CLUSTER`, or `RECORD`, so each run writes the three statistics tables separately. If `outputStats` is not configured, Zingg skips stats writing and the run proceeds normally.

#### JSON

```json
	"outputStats" : {
		"name":"stats",
		"format":"snowflake",
		"props": {
			"table": "zinggStats_$ZINGG_DYNAMIC_STAT_NAME"
		}
	},
```

**Read more:** For the fields in each statistics table → [Output Statistics](/zingg/interpreting-results/explainability-and-statistics/output-statistics.md).

### Step 6: Pass Through (optional)

{% hint style="info" icon="right-long" %}
Pass Through - **Enterprise** only.

Excludes specific records from matching while still including them in output with their own `Zingg ID`.
{% endhint %}

Pass Through excludes specific records from matching while still including them in output with their own Zingg ID. Records matching the passthrough expression appear in the identity graph but never influence cluster formation.

**Note**: Zingg internally applies the negation of `passthroughExpr` to filter matching records. If the passthrough condition applies to nullable fields, ensure the negative of the expression yields the records that are `NOT` passthrough.

#### JSON

```json
{
  "passthroughExpr": "givenname = 'matilda'"
}
```

#### Example with null-safe expression

```json
{
  "passthroughExpr": "is_deceased = true AND is_deceased is NOT NULL"
}
```

{% endtab %}
{% endtabs %}


---

# Agent Instructions
This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com.

## Querying This Documentation
If you need additional information that is not directly available in this page, you can query the documentation by asking a question.

Perform an HTTP GET request on the following URL with the `ask` and `goal` query parameters:

```
GET https://docs.zingg.ai/zingg/running-zingg/configure-zingg.md?ask=<question>&goal=<user_goal>
```

`ask` is the immediate question: it should be specific, self-contained, and written in natural language.
`goal` is what the user is ultimately trying to achieve, the reason they need the answer. Sharing it helps GitBook give you a better, more relevant answer. A goal is most helpful when it describes the outcome the user wants rather than restating the question. For example, with `ask=how do I create an API token`, a goal like `automate deployments from our CI pipeline` lets GitBook tailor the answer to that use case.

The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.