Over the past several years the Xfinity X1 recommendations platform team at Comcast has gone through several end-to-end architectural iterations in order to meet their scalability goals, including rebuilding the data pipeline and the workflow for training and deployment of models. In this talk we discuss our journey, from migrating our data platform from serverless technologies like lambdas to streaming frameworks such as Flink to allow for quick retrieval of data for the runtime components to introducing a For You homepage and what it took us engineer this highly personalized content discovery experience using our A/B Testing platform. We’ll cover some of the many lessons we’ve learned along the way and offer practical advice for those building modern recommendation platforms at scale.
