Snowflake App: Identity Resolution and Enrichment
  • 19 Apr 2024
  • 7 Minutes to read
  • Dark
    Light

Snowflake App: Identity Resolution and Enrichment

  • Dark
    Light

Article Summary

Getting Started

Use Cases


  • Identity and Household Keying

    • Empowering analysis, insights, and distributions at the individual, household, and address levels. Also enables seamless integration of identity completion data and attribute enrichment for enhanced customer profiles.

  • Identity Completion

    • Append the most up-to-date information for each contact, including name, address, phone, and emails, all based on as little as fractional identity markers

  • Attribute Enrichment

    • Complete your consumer profiles by adding more relevant characteristics including demographics, property ownership data, financial characteristics, personal interests, and more.

Prerequisites


  • Access to Snowflake.

  • Necessary permissions to use native apps and call procedures within your Snowflake environment.

  • Select and provide output selections from the VMS identity and attribute data catalog.

  • Requested VMS account activation to be able to use the Enrichment procedure.

Installation


  1. Add Shared App to Your Snowflake Account:

    1. Navigate to the shared apps section in Snowflake.

    2. Locate and install the "Consumer Insights: Identity Resolution and Enrichment" application.

  1. Verify Permissions:

    1. Ensure that you have execute permissions for the function and procedure within the app.

  1. Share Native App Event Logging with VMS:

    1. If Logging and Tracing is not set up in your account you must create and set an Event Table.

    2. Enable logging and event sharing for the application.

Example

SETUP EVENT TABLE

--Create event db/schema
create database <my_event_db>;
create schema <my_event_db>.<my_event_schema>;

--Create event table
create event table <my_event_db>.<my_event_schema>.<my_events_log>;

--Set the account level event table
alter account set event_table=<my_event_db>.<my_event_schema>.<my_events_log>;

--Check active event table
show parameters like 'event_table' in account;

ENABLE APPLICATION LOG SHARING

--Share application log events with provider
alter application <native_app_name> set share_events_with_provider = TRUE;

--Validate sharing flag
desc application <native_app_name>;

The flag share_events_with_provider must be 'TRUE'

  1. Grant Access to Data

    1. Ensure that the location of the input table is shared with the native app. The <database>, <schema> and <table> must be granted to <native_app_name> to allow the keying procedure to access customer data.

Example:

grant usage on database <my_db> to application <native_app_name>;
grant usage on schema <my_db>.<my_schema> to application <native_app_name>;
grant select on table <my_db>.<my_schema>.<my_table> to application <native_app_name>;

 

App Structure


<native_app_name>

|      `--Description: App name defined during installation.

├── APP_DATA

|      `-- Description: Schema to store output tables generated by the application.

└── APP_UTL

    |  `-- Description: Schema that stores native app procedures.

    |     

    ├── REMOVE_TABLE()

    |      `-- Description: Procedure to remove tables created in the APP_DATA schema.

    | 

    └── VMS_ENRICHMENT()

           `-- Description: Procedure to add VMS keys, identity and attribute data.

Shape 

Procedure: VMS_ENRICHMENT

This procedure in the Snowflake Native App allows for the execution of VMS Enrichment process based on a set of defined parameters. Below is the detailed usage of APP_UTL.VMS_ENRICHMENT.

Syntax


To call this procedure, use the following syntax:

call <native_app_name>.APP_UTL.VMS_ENRICHMENT({ 

  'input_table':'<my_db>.<my_schema>.<my_table>', 

  'output_table':'<my_output_table>', 

  'product_name':'<product_name>', 

  'method':'<method>', 

  'name':['<full_name>'], 

  'dob':'<dob>', 

  'address':'<address>', 

  'city':'<city>', 

  'state':'<state>', 

  'zip':'<zip>', 

  'phone':'<phone>', 

  'email':'<email>', 

  'pid_key':'<vms_pid>', 

  'hhid_key':'<vms_hhid>' 

});

Arguments


<input_table>: The name of the input table in the format <database>.<schema>.<table>.

Example: 'my_marketing_data.leads.lead_db'

<output_table>*: The name of the desired output table. If this parameter is omitted, for keying the output table name will be created by using the name of the input table plus the prefix _VMSKEYS and for enrichment the output table name will be created by using the name of the input table plus the prefix _ATTRIBUTES.

Example: 'leads_output'

<product_name>: The name of the product configuration available for the customer. This value will be provided by VMS after setup and configuration agreement.

Example: 'TCI Standard'

<method>: Indicates the process type to be undertaken through the VMS_ENRICHMENT procedure, can be one of the following options:

  • 'keying'

    • Process customer data by using VMS identity resolution intelligence, returning identified VMS keys and providing match level details.

  • 'enrichment'

    • Process and return the identity and attribute data related to the <product_name> based on the specified VMS Keys <pid_key> and <hhid_key>.

  • 'plus'

    • Performs both keying and enrichment processes on the same procedure call. The ouput of this process will create two output tables, one with VMS keys and one with VMS keys plus attributes. 

<name>: An array containing the column name(s) for the full name. 

Examples: ['full_name'] , ['fname','mname','lname'] , ['fname','lname'] 

<dob>*: Column name for the date of birth.

Example: 'dob'

<address>: Column name for the address.

Example: 'address'

<city>: Column name for the city.

Example: 'city'

<state>: Column name for the state.

Example: 'state'

<zip>: Column name for the ZIP code.

Example: 'zip'

<phone>*: Column name for the phone number. 

Example: 'phone'

<email>*: Column name for the email address.

Example: 'email'

<pid_key>†: Column name that contains the VMS PID Key to be used during the enrichment and plus processes.

Example: 'PID'

<hhid_key>†: Column name that contains the VMS Household ID Key to be used during the enrichment and plus processes.

Example: 'HHID'

*Optional

*The fields output_table, dob, email and phone are optional for the keying process setup.

Household ID

†Either pid_key or hhid_key can be selected for an enrichment or plus process configuration. If both are specified, the process will only return related Household ID information when individual data is not available for the provided PID Key.

Examples


VMS Keying:

--VMS Keying process call

call <native_app_name>.APP_UTL.VMS_ENRICHMENT({

  'input_table':'marketing_db.leads.customers',

  'output_table': 'customer_output_01',

  'product_name':'tci attributes',

  'method':'keying',

  'name':['full_name'],

  'dob':'dob',

  'address':'address',

  'city':'city',

  'state':'state',

  'zip':'zip',

  'phone':'phone',

  'email':'email'

});

This call will execute the keying process with the specified parameters, using the data from the specified <input_table>, and create an output table named app_data.customer_output_01. The output table will include the input data along with the associated VMS Keys found and assigned match level.

VMS Enrichment:

--VMS Enrichment process call

call <native_app_name>.APP_UTL.VMS_ENRICHMENT({

  'input_table':'app_data.customer_output_01',

  'output_table': 'customer_output_01_en',

  'product_name':'tci attributes',

  'method':'enrichment',

  'pid_key':'PID'

});

This call will execute the procedure with the specified parameters, using the data from the specified <input_table> with VMS Keys and create a table named app_data.customer_output_01_en. The output table will include the input data along with the associated TCI attribute data found for each VMS PID.

VMS Plus:

--VMS Plus process call

call <native_app_name>.APP_UTL.VMS_ENRICHMENT({

  'input_table':'marketing_db.leads.audience_data',

  'output_table': 'audience_data_insights',

  'product_name':'tci attributes',

  'method':'plus',

  'name':['full_name'],

  'address':'address',

  'city':'city',

  'state':'state',

  'zip':'zip',

  'phone':'phone',

  'pid_key':'PID',

  'hhid_key':'HHID',

});

This call will execute the keying and enrichment process with the specified parameters, using the data from the specified <input_table>, and create an output table named app_data.audience_data_insights. The output table will include the input data along with the associated VMS Keys and TCI attributes found based on linked VMS PIDs and Household IDs.

 

 

Output


NOTE: Snowflake Native Applications write data to tables stored within the application. To copy data to your databases, be sure to replicate the outputs after the application has run. Any time the application is run or updated to a new version, any data currently stored in the application will be overwritten or deleted, respectively.

VMS Keying Output:

  1. MATCH_LEVEL

  • Description: Indicates level of identity match to the VMS Graph

  • Type: Integer

  • Details:

  • 1 - Indicates a match at the individual level

  • 2 - Indicates a match at the household level

  • 3 - Indicates a lower level match

  • 4 - Indicates a match to a single identity marker where multiple markers were provided

  • 5 - Indicates a match to an identity that does not have demographics available – gender, age, and dob are the only fields that may contain values

  • 6 - Indicates that no match was made our identity graph but a gender could be inferred based on analysis of the name present on input

  1. PID (Personal Identifier)

  • Description: A unique identifier assigned to each individual in the dataset.

  • Type: Encrypted string

  • Purpose: Used to uniquely identify and track individuals across different VMS datasets and solutions.

  1. HHID (Household Identifier) 

  • Description: A unique identifier assigned to a household.

  • Type: Encrypted string

  • Purpose: Helps in aggregating and analyzing data at the household level, enabling insights into household-based trends and patterns.

  1. ADDRID (Address Identifier) 

  • Description: A unique identifier associated with the physical address specified.

  • Type: Integer

  • Purpose: Useful for geographical analysis and location-based insights. Assists in distinguishing between individuals/households at the same or different locations.

VMS Enrichment Output:

  1. KEY_TYPE

  • Description: Indicates the type of keying match used to return VMS Identity and Attribute data:

  • Type: String

  • Details:

  • P - Indicates that the returned data corresponds to the specified VMS PID Key.

  • H - Indicates that the returned data corresponds to the specified VMS HHID Key.

  1. VMS_ATTRIBUTES

  • Description: Contains the returned attribute data found for the specified VMS Key.

  • Type: JSON

VMS_ATTRIBUTES Example:

{ 

  "ADDRESS": "555 VMS DR E", 

  "ADDRID": "9999999999", 

  "AGE": 45, 

  "CBSA": "27260", 

  "CENSPCT_WATER": 4, 

  "CENSUSBLCK": "2006", 

  "CENSUSTRACT": "016000", 

  "CENS_POP_DENSITY": 2678, 

  "CENS_POP_MEDAGE": 36, 

  "CHARITYDNR": "Y", 

  "CHILD": "Y", 

  "CITY": "JACKSONVILLE", 

  "CPI_APPAREL_INDEX": 8, 

  "CPI_BUSINESS_HOMEOFFICE_INDEX": 0, 

  "CPI_CRAFTS_INDEX": 2, 

  "CPI_HOMEDECR_INDEX": 7, 

  "CPI_OUTDOORS_INDEX": 4, 

  "CPI_TRAVEL_INDEX": 6, 

  "CREDITCARD": "Y", 

  "CT_DNR_CONTRIB_ENVIRO": "Y", 

  "CT_DNR_CONTRIB_HEALTH": "Y", 

  "DOB": "197902", 

  "DPC": "179", 

  "DPV": "Y", 

  "EHI": "E", 

  "EMAIL": "JOHN.DOE@MYEMAIL.COM", 

  "FNAME": "JOHN", 

  "GENDER": "F", 

  "GEOLEVEL": "01", 

  "HHNBR": 4, 

  "HOMEOWNERCD": "H", 

  "HOUSE": "555", 

  "LATITUDE": 41.303921, 

  "LNAME": "DOE", 

  "LONGITUDE": -81.901693, 

  "MNAME": "B", 

  "OCCUPATIONCD": "K", 

  "PHONE": 4443332211, 

  "POSTDIR": "E", 

  "STATE": "FL", 

  "STREET": "VMS", 

  "STRTYPE": "DR", 

  "YRBLD": "1973", 

  "ZIP": "32216",  

} 

Shape 

Procedure: REMOVE_TABLE

This procedure in the Snowflake Native App allows for the removal of tables create by the app on the APP_DATA schema.

Syntax


To call this procedure, use the following syntax:

call <native_app_name>.APP_UTL.REMOVE_TABLE('<table_name>');

Arguments


  • <table_name>: Name of the table to be removed on the APP_DATA schema of the native app. 

  • Example: 'my_table_name

Example


call <native_app_name>.APP_UTL.REMOVE_TABLE('consumers_vmskeys');

Data Integrity and Privacy


  • Confidentiality: All identifiers and returned data are treated with strict confidentiality and are compliant with data privacy laws and regulations. 

  • Accuracy: The procedure ensures the accuracy of these identifiers to maintain data integrity. 

  • Encryption: The PID and HHID are encrypted using AES-CBC (Advanced Encryption Standard - Cipher Block Chaining) to ensure additional per customer security and privacy. 

Support


For any issues or questions regarding the app, please contact the support team at Verisk Marketing Solutions Support

 


Was this article helpful?