Enrichment

​

Overview

Your own data can only get you so far. It’s common for data pipelines to enrich first party data with third party data in order to get a more complete picture. This is usually a complicated engineering task and involves running additional infrastructure but in Vinyl enrichment is supported directly in the transform layer.

Common use cases for enrichment include:

• Zipcode to city mapping
• IP address to location mapping
• Email to company mapping
• Phone number to company mapping
• Weather enrichment

We support enrichment through managed data broker services. These services are provided by third party vendors and are integrated directly into Vinyl. You can bring your own API key. Some initial providers that we support in private beta are:

• Apollo
• Clearbit
• WeatherAPI
• ZipcodeAPI

​

How to Enrich Data

Enriching data is as easy as passing in the provider to your existing model.

Let’s say you have a SaaS product postgres database with users in it that you’d like to enrich with third party data from Apollo.

We can use Vinyl’s ApolloEnrichment provider to enrich the data for matching columns and transform the data.

from vinyl import model
from vinyl.enrichment import ApolloEnrichment
from my_project.sources import PostgresProductUsers

@model(deps=[PostgresProductUsers, ApolloEnrichment])
def enriched_users(users, apollo):
    users.enrich(apollo, on='email', cols=['company', 'title', 'website', 'location', 'size'])
    return users