So, You Want to Move to Microservices?

As I reflected back on my time at LinkedIn, I put together a brief history of its scaling story. We had done the (now) classic migration from monolith to microservices. Just like oh I dunno, Amazon, Google, eBay, Twitter, Netflix, and my current employer Uber (to name a few). And why not? Microservices are a great way to scale your traffic as you scale your engineering headcount.

So clearly every company should do it, right?

Microservices absolutely have advantages, but everything in life is a tradeoff. Let’s dive into some of the pros and cons of microservices. And if you’re still convinced, let’s tackle what you need to get right before embarking on the microservice journey.

Microservice Pros and Cons

Pros:

• Easy to horizontally scale (if the service is stateless). Just deploy more and have the load balancing take care of the work.

• Easy to understand. Services typically have a smaller codebase, with limited functionality, and a clear purpose. Much easier to maintain and extend.

• Great for teams. They can own their own release and deployments. This can improve resiliency, uptime, and overall degradation. They can also enjoy some technical freedom if they chose a different language.

Cons:

• Serious operational overhead. Deploying every service at random times every day adds up. You’ll need a full-time team.

• Backwards compatibility issues. Introducing new APIs can be tricky. Feature flagging is a must. What version is that backend? What if it needs to roll back?

• Complex call graphs and fanouts make debugging hard. Instead of navigating a single repo, are you jumping around, digging through logs, to find the bug?

• Network complexity. Every call now has to marshal / unmarshal, go over a network connection that’s still going to drop some packets, and that only compounds with a complex call graph. Some random latency in your fanout could cause a ripple effect and take down the site.

• Possible duplication because of human nature. Humans tend to avoid conflict, so microservices make it that much easier to avoid others and just build a new service. Often at the cost of overall efficiency.

So, what should I do?

Those cons above are nasty and represent only a glimpse of the world of microservices. You don’t want to get into that too early. It’s better to start with a monolith as your company or team is small and move toward microservices when it becomes painful to add new features or scale your system.

Almost all the successful microservice stories have started with a monolith that got too big and was broken up — Martin Fowler, Monolith First

In LinkedIn’s case, the first “service” in their move towards microservices was one that maintained their member connection graph. The usage profile was different from other parts of the site, they wanted it to live in-memory for performance, and even their DB queries were different. So it made sense to be standalone.

What do I need to get right before embarking on a microservice migration?

In short, you need to understand and address the tradeoffs you are making in the “cons” of microservices. Here’s a list I often think about:

• Observability / Monitoring / Alerting

• Feature flagging to enable dark launches, guard against compatibility issues, and rollback faster if things go wrong.

• A timeline tool that tracks deployments, outages, feature flag rollouts, config changes, etc.

• A deployment tool that shows every host and the version currently deployed to it

• A robust CI / CD environment

• Distributed logging

• A tool to visualize each service’s dependencies (downstream and upstream)

• A distributed tracing tool to help more complex debugging

• A shared understanding of good API design. Optionally auto-generated documentation of all known APIs with usage examples.

• An internal repository to store and share things like shared libraries

• Service discovery infrastructure

• Dealing with network inevitabilities: re-tries, exponential backoffs

• Shared libraries for graceful degradation and resiliency: load-shedding, circuit-breaking, rate-limiting

• Testing infrastructure: unit testing, load testing, chaos testing, integration tests, automation tests

Conclusion

Doing microservices right is hard (sadly, many end up with a distributed monolith instead). Start with a monolith especially if the team is small. Find that successful product market fit. The time will present itself. Start with one new service. Get the tooling / infrastructure in place. Then add more. Eventually you’ll have the confidence to migrate the rest.

Update Nov 2016: A nice video of the realities of migrating to microservices from one of Uber’s lead engineers mirrors many of these points.