Micro Services Architecture

In programming we decompose big things into small, manageable chunks. In algorithms, we solve a small problem, and then use recursion or iteration to solve more complex problems. In software development, we create methods, and interfaces, and classes, and packages with the objective of being more efficient, and letting our code be more maintainable, manageable…so that a change in one part does not require a cascading change in other parts of the application.
Yet, we deploy as a monolith. All the packages run in the same JVM, in the same process. Classic example of this is the Java EE container, where all the wars, jars, and ears are deployed in the same JVM. In this page, I am jotting my notes on micro services, and a high level framework for designing micro services.

Challenges of a Monolith

Deploying all logical units into the same physical space introduces a lot of challenges:
  1. What happens when there is a memory leak in one of the classes of the monolithic application? The entire app goes down on knees. This brings out the Reliability challenges.
  2. Finding the memory leak in a monolithic app brings up some significant Supportability challenges.
  3. Serviceability challenges: In continuos integration, and devops model, you have to wait for low traffic windows for upgrades, etc.
  4. Testing challenges: What functional areas need to be regression tested when 1 class (out of 20k classes) changes? With a big ball of mud, who knows what functions get affected. With plenty of automation, that might be okay, but not everybody has that.
  5. Availability challenges: Take forever to start.
  6. Maintainability and Team Growth challenges: Too big to understand. Also has team challenges, and its difficult to scale the team quickly to work on a significant codebase. Imedes the team velocity.
  7. Productivity challenges: Takes forever to build. The edit, compile, debug, test cycle is frustrating. (JRebel!)

Micro Services

“Micro services” is an architectural approach of designing a product as a suit of services that work together. Each service could use a different programming language, technology, or could be running in the same or a different physical host. Its inspired with the design of Unix, where a set of small programs do wonders when their outputs are piped/tee’d. All infrastructure items are good candidates for micro services.
Desired attributes of a micro service:
  1. Single Responsibility Principle – Micro service is a software to do a small thing, and do it well.
  2. Replaceable, with well defined interfaces – Micro services must be replaceable as technologies change. The overall product never becomes a legacy software.
  3. Runs in its own process – this allows for figuring out resource requirements on a service basis, and scaling correctly. You can measure demands of a specific process, and scale that process only if needed.
  4. Easily consumable (HTTP/JSON/ProtocolBuff etc.)
  5. Free of temporal coupling – We don’t want service X to be deployed before service Y.
  6. Must be isolated, decoupled, and encapsulated.
  7. Testable.
  8. Fast and easy to startup.
Twitter, Netflix, and Gilt have transformed their monoliths into hundreds of micro services that talk to each other to perform any function.

Not a Free Lunch

There are many challenges associated with micro services:
  1. Deployment Complexity – With dozens of micro services, you are looking at a lot of deployments. Physically shipping loads of micro services  into production can only be done through a baked release and deployment automation process. Thus, only organizations invested in DevOps must venture in this area. However, deployability is one of the benefits of micro services as well, because you can do canary releases of each service, and push even smaller changes into production, avoiding downtimes. With services designed without temporal couplings, each service can be deployed separately. Think of tools like Jenkins, Puppet, Chef, Ansible, that can help here.
  2. Operational Complexity – With many processes to manage and monitor, you have to write monitors for separate processes. You will need to pinpoint the bottlenecks in a system where multiple services are talking to each other. Aggregation and visualization of performance monitoring data is critical for this. Use tools like Graphite, Ganglia for data aggregation of each micro service. Have a micro service for “synthetic monitoring” of other micro services. Have defined synthetic monitoring end points in each of your micro service.
  3. Log analysis: Looking into log files of many different micro services is not scalable for operations. Use tools such as log stash, kibana or splunk to get an aggregated and indexed view of logs. One service makes a call to another, which then makes a call to another. Use a “correlation id” across services, that can be used to track related requests across micro services.
  4. Fault resilience – Each service must be written in a fault resilient manner. Complexity displacement – from logical space to physical space.
  5. Increased overall memory consumption.
  6. Increased latency, as the in-memory invocations get replaced with RPC calls.
  7. Convincing the stakeholders to invest in multiple micro services could be challenging.
  8. Micro services allow to use whatever language/platform that you are comfortable with for each micro service. However be careful of being polyglot, as there is a cost for support/training on different languages.

High Level Framework for Designing a Micro Service

Break down the Monolith

The biggest challenge is about breaking down the monolith, and finding the boundaries of a micro service. Defining a bounded context for each small service can be daunting. For an existing monolith, carve off micro services carefully, and do it in small steps. If you do not want to get to the extreme end of micro services, just target the infrastructure services first – like monitoring, messaging, file sharing, authentication, etc.

Domain Driven Design complements a micro service based architecture. Use single responsibility principle for defining the boundaries.

Improve Resilience by Minimizing Couplings

Failure is inevitable with a lot of micro services, and a cascading failure can bring down every thing. Your app must be able to tolerate failures. Availability decreases with dependencies, and increases with redundancy.
Inspect the micro services to see if:
  1. You can avoid synchronous RPC calls. If you feel this is needed, look again at the boundaries, and see if synchronous calls can be avoid by adjusting the boundaries.
  2. If you see that boundaries are correct, and you still need synchronous RPC calls, then consider a circuit breaker (Look at Netflix Hystrix). Its expensive to cascade failures to other consuming services. The circuit breaker allows to fail fast, and lets the consuming service proceed. Use tryAcquire semaphores, thread pools, and aggressive network timeouts for opening circuits, and  alerting operations. The micro services must be allowed to define the behavior in case of failures. However, the behavior is implemented in circuit breakers.
  3. Prefer a async pub-sub model, with something like Kafka as MOM inter mediatory (another service!!)

Securing the Micro Services

Keep the micro services behind a proxy. The proxy acts as a circuit breaker for failed services. Each service goes through the proxy to talk to the other service. Other things that can be done in proxy are traffic shaping, ACL’ing, load balancing, active monitoring, and attack protection (if service is accessible outside).


  1. Strive for a shared nothing architecture. Let the services have their own lightweight databases. If sharing database, use separate schema.
  2. Plan a regression test suit for each micro service.
  3. Design a built-in diagnostic and monitoring within each micro service
  4. Deploy through canary release.


Leave a Reply

Your email address will not be published. Required fields are marked *

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>