Delivering personalised content at scale with Smart Links - Part 2
In part 1, we covered the challenges in personalizing visuals at scale and how teams can use Smart Links to solve these challenges. Let us switch gears and dive into the technical details of Smart Links. We will cover:-
Problem Statement: Find the
k most relevant variants from the
n variants in a project
n creative URLs, each with a set of
m key-value pairs of metadata
k most relevant creative URLs
We built a modified version of
K-nearest neighbors to solve the search problem. Based on the input key-value pairs, we generate a score between
0-k for every variant inside the project. We then sorting the variants in descending order of the generated score and pick the first
O(n * m)
Let us start with the 4 variants we looked at in part 1.
age group: 30-45,
age group: 20-30,
age group: 20-30,
location: San Francisco,
age group: <20,
Let us say we are searching for a creative with
gender: male and
purchases: 0-2. We start with a base score of 0 for all the variants. We match the given key value pairs with each variant’s metadata. Each match is 1 point.
Variant 1: Gets 1 point as it matches
Variant 2: Gets 1 point as it matches
Variant 3: Gets 0 point as it does not match anything
Variant 4: Gets 2 points as it matches
When we sort by the score, variant 4 comes out on top with a score of 2.
There are 2 parts to this architecture:-
Rendering - Creatives are rendered using a combination of Lambda (serverless) and auto-scaling EC2. They are stored in S3 with the URL saved in a MongoDB database.
Delivery - The API uses CloudFront, load balancer, Lambda, and Redis.
Is integrated with Lambda using Load Balancer as there is no direct way to connect Lambda and CloudFront
Is used to cache JSON response and/or binary creative data
JSON responses are cached against query parameters (for sending search attributes) and headers (for authentication)
Checks for same request in the nearest edge location, reducing overall latency drastically
Changes made to project in Rocketium triggers an invalidation request in CloudFront distribution so users always have the latest content
Just acts as a middleware to connect Lambda and CloudFront
Mapped as an origin inside respective CloudFront distribution
A behaviour needs to be defined for endpoint
/api/v1/creativesto redirect request to mapped load balancer
Avoids having to manage dedicated servers and scaling
Uses search algorithm defined above to find the most relevant creatives
Uses reserved concurrency to save cold start time and improve latency
Data retrieval is much faster from Redis than a database
In case of miss, variant metadata is fetched from MongoDB and cached in Redis
Lambda checks for a hit in Redis first before making a database call
In these two parts, we learned about the why, how, and what of Smart Links. Coming soon is Cortex, a layer on Smart Link which harnesses its power to run personalised in-app campaigns. Until then, see it in action at https://rocketium.com/is/in-app-campaigns/.