Engineering the New LinkedIn Profile

This post I originally wrote on the LinkedIn Engineering Blog that documents my team’s journey rewriting LinkedIn’s main member profile page. It’s a combination of a brand new design, numerous new product features, and a brand new frontend architecture. This architecture uses a custom Java web application framework called Unicorn as well as using the Dust.js javascript template engine.

---

Last year, we introduced a major redesign to the LinkedIn profile. Not only did we update the look and feel, but we added new features to the product in order to better showcase your professional identity.

Blah vs. Huzzah!

When considering the number of new features, the Profile engineering team realized we needed to completely rethink the technologies used for the page. We ended up adopting a brand new frontend architecture.

In this post, I’ll cover our use of 3 new technologies - JSON mappers, Fizzy, and dust.js - and how we used them to create a more fluid and dynamic page, while maintaining high performance.

New Frontend Architecture

Traditionally, the HTML of the profile page was rendered on our servers using JSPs within the Profile web app.

With the New Profile architecture, we introduced 3 key changes:

1. Our Profile web app uses Mappers to serve up JSON data for different sections of the page.
2. A system called Fizzy asynchronously fetches and aggregates this data.
3. Dust.js client templates render the HTML within the browser.

Mappers - JSON endpoints

Instead of using JSPs on the server and returning HTML to the browser, the Profile web app has been repurposed to serve the data for the page as JSON.

We created classes called Mappers in our web app that combine, adapt, or modify data from our backend services into JSON. Each Mapper has an independent URL endpoint and exposes JSON data for a specific part of the page. It retrieves that data from any number of downstream services.

The Profile web app has Mappers for each section of the page: positions, summary, connections, followed companies, and so on. We can make parallel requests for these endpoints individually or, alternatively, make a batch request for a number of like-minded endpoints. For example, for the Top Card, we batch a request for JSON data from the picture, positions, educations, and connection count Mappers.

The advantage of this structure is that we can reuse Mappers in different parts of the page: for example, the positions Mapper is used in both the Position section and Top Card. We can also hit Mapper endpoints directly via AJAX when dynamically refreshing a section on the page.

Fizzy - UI aggregator

What’s responsible for fetching these Mapper endpoints and rendering the data? Enter Fizzy - our homegrown UI aggregator layer.

Fizzy is able to fetch and aggregate UI components served from one or more web applications and render them on a single page. It consists of two parts:

1. Fizzy server-side: the server part of Fizzy is an Apache Traffic Server (ATS) plugin. ATS proxies all requests and responses to the site. As an ATS plugin, Fizzy can inspect every web server’s response, and process special directives in that response (described below), before flushing data down to the browser.
2. Fizzy client-side: the browser part of Fizzy is a JavaScript library that is responsible for rendering the markup when data from each web app is received.

How do web apps tell Fizzy what to do? We first define a base HTML page with multiple “embeds” for each UI section on the page:

We break down the Profile page into UI sections called “embeds”

Looking at the code, the base page is largely an empty skeleton, with special markup that acts as a placeholder, telling Fizzy what data to embed into the page:

Fizzy will parse this information and fire off a parallel request to the endpoints specified. When data is returned, Fizzy will inject the data and its own markup into the base page, flush it to browser, and use the Fizzy client side code to progressively render the page.

Deferred Rendering & Fetching

By breaking down the Profile page into small UI embeds, we can improve the performance of the page by optimizing for above the fold.

You’ll notice the New Profile may fetch and render the Top Card first, while fetching the sections at the bottom of the page much later. This is especially helpful for long profiles. Our goal was to render the above the fold page as fast as we can - improving page load times compared with the old JSP-based profile.

Dust Client Templates

The New Profile is LinkedIn’s highest traffic page to use client side templates. As described before, we chose Dust.js to replace JSPs as our new templating technology.

The decision to use client templates for the New Profile made a lot of sense. First, users typically view profiles in batches. Therefore, being able to cache the static markup and have our servers just return the dynamic profile data was a big win for performance. Second, it allowed the page to come alive by permitting users to paginate through modules, perform inline searches for connections, and easily update their profile using inline edit forms. By using client templates (with our separate JSON endpoints), we can easily re-render a section and update the markup in-place.

Re-render and Replace!

The ease of re-rendering templates enabled our page to become more interactive. One of the better changes over the old profile was the ability to inline edit every form on the page. We wanted to make it as seamless as possible to modify any item on your profile.

For example, let’s consider the Summary section. We created multiple templates including an outer container with edit controls, a “view” template, and an “edit” template representing the form.

Each template will construct the form, prefill the data, and add JavaScript to handle the Ajax form submission. Upon submit, the server returns the updated summary data as JSON. This triggers a callback to re-render the “view” template with the returned JSON.

Using Dust to render this HTML improved our development speed and provided a repeatable pattern for all our edit forms.

New technology, new techniques, new profile

And there you have it: by leveraging JSON endpoints, Fizzy, and Dust templates we managed to stitch together a slick user experience containing nearly twice the content without sacrificing performance. If you haven’t already, make sure to check out the new profile and let us know what you think!

For an even more in depth look at the technology behind the profile page, check out the following presentation:

« PREVIOUS New App Engine Pricing may kill Earthquake Alerter

NEXT » Draft Punk for 2013