Open Sourcing Brooklin: Near Real-Time Data Streaming at Scale

Brooklin - a stateless and distributed service for streaming data in near real-time and at scale - has been running in production at LinkedIn since 2016, powering thousands of data streams and over 2 trillion messages per day. Today, we are pleased to announce the open-sourcing o … | Continue reading


@engineering.linkedin.com | 4 years ago

Using Virtual Private Clusters for Testing for Real Clusters

If Apache Kafka is the lifeblood of all nearline processing at LinkedIn, then Apache Samza is the beating heart pumping that blood around. Samza at LinkedIn is provided as a managed stream processing service where applications bring their logic (leveraging the wide variety of Sam … | Continue reading


@engineering.linkedin.com | 4 years ago

Improving Performance and Capacity for Espresso with New Netty Framework

In this blog post, we’ll share how we migrated Espresso, LinkedIn’s distributed data store, to a new Netty4-based framework and achieved a large performance and capacity gain throughout the Espresso system as a result. In the larger scheme, this is particularly important since Es … | Continue reading


@engineering.linkedin.com | 4 years ago

Lighter Than Lightweight: We Built the Same App Twice with Preact and Glimmer

The beauty of the web is that there is no “install” step. Someone, somewhere taps a link to your site, and moments later it appears instantly in front of them. | Continue reading


@engineering.linkedin.com | 4 years ago

Detecting Interference: An A/B Test of A/B Tests

At LinkedIn, most decisions are made using experiments. When we want to decide between two features, we test them against each other in the real world: we give feature A to a random set of members, feature B to another set, and we compare the results. Are users of feature A more … | Continue reading


@engineering.linkedin.com | 4 years ago

Driving Business Decisions Using Data Science and Machine Learning

Co-authors: Burcu Baran, Xiaojing Dong, Chi-Yi Kuan, Emily Huang, and Tiger Zhang   | Continue reading


@engineering.linkedin.com | 5 years ago

Privacy-Preserving Analytics and Reporting at LinkedIn

Co-authors: Krishnaram Kenthapadi, Thanh Tran, Mark Dietz, and Ian Koeppe | Continue reading


@engineering.linkedin.com | 5 years ago

Authorization at LinkedIn’s Scale

LinkedIn members entrust us with their personal data and we are committed to working hard every day to maintain that trust within a safe, professional environment. One crucial aspect to earn and maintain that trust lies in how well we secure our online systems and protect our dat … | Continue reading


@engineering.linkedin.com | 5 years ago

The Growth of IPv6 in the Asia Pacific Region

IPv6 was created to increase the number of available devices directly connected to the internet from 4 billion to too many to count by increasing the address space from 32bits to 128bits. In some ways, IPv6 was made for countries that will need to bring millions of users to the i … | Continue reading


@engineering.linkedin.com | 5 years ago

Scaling Machine Learning Productivity at LinkedIn

Co-authors: Joel Young, Bee-Chung Chen, Bo Long, Marius Seritan, and Priyanka Gariba   | Continue reading


@engineering.linkedin.com | 5 years ago

Interviews on Replacing Infrastructure Systems at LinkedIn

LinkedIn’s infrastructure has grown remarkably over the years, and that has kept the Production Infrastructure Engineering teams on their toes. Growing quickly means we’ve had to learn to adapt quickly. While most of our systems are solid and ready to scale, from time to time we  … | Continue reading


@engineering.linkedin.com | 5 years ago

LinkedIn winds back functionality of its APIs

Continue reading


@engineering.linkedin.com | 5 years ago

Defending Against Abuse at LinkedIn’s Scale

LinkedIn is committed to building a safe, trusted, and professional environment. Building the infrastructure to detect and mitigate abuse at LinkedIn’s scale brings with it a number of interesting challenges that require us to often think outside the box and craft new ways of int … | Continue reading


@engineering.linkedin.com | 5 years ago

Building Representative Talent Search at LinkedIn Engineering

Co-authors: Sahin Cem Geyik and Krishnaram Kenthapadi   | Continue reading


@engineering.linkedin.com | 5 years ago

Building Enterprise Software on LinkedIn's Consumer Stack

Co-authors: Viman Deb and Christian Mathiesen   | Continue reading


@engineering.linkedin.com | 5 years ago

Coding conversations: Transforming my career

Transformation. Is there a right time to transform your career? Is there a cheat sheet that helps us transition from role A to role B? How do we figure out what role we are passionate about? How do we figure out the right time to transform? | Continue reading


@engineering.linkedin.com | 5 years ago

LinkedIn Lite

The opportunity in India India is a mobile-first country, with 71% of the population having only a mobile internet connection and accessing the internet only via mobile.  Also, 85% of the mobile population accesses the internet on an Android device, with UCWeb being the preferred … | Continue reading


@engineering.linkedin.com | 5 years ago

Making LinkedIn's Organic Feed Handle Peak Traffic

LinkedIn has recently experienced issues with handling peak load in FollowFeed, our custom timeline service that powers the organic updates members see in their activity feed. Examples of organic updates are text/article/image shares from members you are connected to, updates abo … | Continue reading


@engineering.linkedin.com | 5 years ago

Adopting a growth mindset for engineers

Contrary to popular perceptions, I’ve come to realize engineers do not exclusively work on building new products. Engineering sometimes involves designing new systems, but can also entail evolving older legacy systems, scaling products, and maintaining good hygiene by cleaning up … | Continue reading


@engineering.linkedin.com | 6 years ago

The Log: What every engineer should know about real-time data abstraction (2013)

I joined LinkedIn about six years ago at a particularly interesting time. We were just beginning to run up against the limits of our monolithic, centralized database and needed to start the transition to a portfolio of specialized distributed systems. This has been an interesting … | Continue reading


@engineering.linkedin.com | 6 years ago

CSS at Scale: LinkedIn’s New Open Source Projects Take on Stylesheet Performance

Browsers use Cascading Style Sheets (CSS) to control the appearance of websites. From borders, fonts, and colors to layout, images, and animations, there are roughly 500 different style properties that can be declared with CSS. These properties are what make the visual diversity … | Continue reading


@engineering.linkedin.com | 6 years ago