We all know what microservice is now but how does service mesh assist with microservice development.
Microservice
Microservice as an architecture was firstly conceptualized in this article by Martin Fowler in 2014. It covers the pros (strong module boundaries, independent deployment, technology diversity) and cons (dealing with distributed system, eventual consistency, operational complexity). The reality is, many teams develops their product with the microservice architectural pattern.
The implementation of microservice architecture involves a lot of programming patterns, and tools. The creation of these patterns and tools are usually done in a separate dedicated project so developers can focus on business logics. When building software, developers only need to interact with libraries and frameworks. Libraries (e.g. log4j) provides dependencies, and developers needs to write code to call the libraries. On the other hand, frameworks (e.g. Spring, Flask) not only provides tools, but also implements a pattern. It addresses a set of common problems such as authentication, expose http service, logger and database connectivity. Once set up, the framework will call the code that developers write (unlike in libraries).
Spring – an example of Microservice framework
When it comes to microservice, a well-known appliction framwork is the Spring framework. It solves problems such as:
- Application context and dependency injection (for Inversion of Control, or IOC)
- Database access and transaction management
- Expose rest APIs (using spring MVC)
There’s an entire ecosystem of projects under Spring framework. This framework is a huge system requiring a lot of configuration efforts. This is where Spring Boot helps.
Spring Boot makes it easy to create stand-alone, production-grade Spring based applications that you can just run. It features the “convention over configuration” paradigm to save programmers from boiler plate configuration. SpringBoot gives you a standalone application ready to run without complicated deployment steps.
Managing configuration in property files does not scale in the time of microservice. Spring Cloud provides configuration as a service (in line with everything else microservice framework). It doesn’t necessarily have to be hosted in the cloud. It is comparable to Apachee Zookeeper, Etcd (distributed key value store), Hashicorp Consul and Netflix OSS (Eureka, Ribbon, Hystrix). You can pull from Git repo. The mission of Spring Cloud is to eliminate boilerplate associated with distributed systems problems for Spring Boot applications.
Many developers use Spring Boot along with Spring Cloud to build microservices. This page contains a diagram for such architecture.
In this architecture, Spring cloud helps with service discovery, traffic routing, circuit-breaking, distributed tracing and monitoring. It can also act as API gateway (in place of Nginx).
API Gateway and API Management
Microservices relies on API. Let’s distinguish API gateway, and API management (this long post has some good information).
API Gateway is a microserivce pattern. The idea is a single point of entry for all clients. The API gateway either proxy an incoming request to the appropriate service, or it may fan out a request to multiple services. The other important aspect is the API gatway can expose a different API for reach client.
A variation of this pattern is the Backends for frontends pattern, where it defines a separate API gateway for each kind of client. The API Gateway may authenticate user and pass an Access Token containing information about the user to the services. It may use a circuit breaker to invoke services. To summarize, the key functions of an API Gateway in this pattern is:
- Unified entry point for multiple API implementations
- Protocol transformation
- Request morphing
- Client specific logics
This page from Azure has a good comparison between API Gateway pattern vs direct connection between client and microservice. Note that API Gateway can also refers to API Gateway product, which implements the functions above. For example
- Spring Cloud Gateway
- Solo.io Gloo
- Netflix Zuul
API management acts as a proxy for an existing API implementations. Typical functions include:
- AuthN and AuthZ
- Service Discovery Ingreation
- Load balancing (e.g. L7 path based routing)
- Logging, tracing (track user), correlation
- Response Caching
- Retry policies, circuit breaker, QoS
- Enforce policy
- Track usage and monetization
- metrics (duration)
- rate limiting and throttling
- Request morphing (header, query string and claims transformation)
- IP whitelisting
API management are usually implemented as tightly controlled shared infrastructure owned by either a “platform team”, “integration team”, or other API infrastructure teams. Examples of API management product (including SaaS) are:
- Google Cloud Apigee
- Mulesoft
- Kong
In real world, people often use API management produce and API gateway product interchangeably.
Service Mesh
Many dub Service Mesh the next generation of Microservice. So what is the relationship between microservice and service mesh. The Microservice architectural pattern creates the need for API gateway pattern. To address this pattern, the API Gateway products first emerged. Service mesh emerged later. Service mesh and API gateway have a common set of features.
In this presentation (a tale of two frameworks) from 2018 (early days of service mesh), two teams discussed microservice (spring cloud) and service mesh (istio) approaches. There is a slide about when to use which. Many teams since have moved to Service Mesh for feature richness. This is a case study from 2021.
There are a number of service mesh technologies, such as Consul, Isito and Linkerd. Here is a comparison chart. Even though all those projects are open-source, there is some competition already. Linkerd is the first to bring up the concept of service mesh in this blog. It also purportedly has better performance than Istio. However, it does not use Envoy proxy. Istio is good at marketing. It has higher adoption rate and is feature rich. However, Google did not donate Istio project to CNCF as many expected. Instead, it created its own governing body, the Open Usage Commons. The Istio is not an open-governance project, which potentially diverge from CNCF in the future [1]. Hashicorp Consul initially was built for service discovery and distributed key/value store. It supports Kubernetes and VM. However, it still lacks observability features.
Apart from the three major technologies, other players tries to push for standardization of service mesh. The most influential initiative is the SMI (service mesh interface), pushed by Microsoft. The idea is a separation of standard and implementation, so late players will have a chance. OpenServiceMesh is Microsoft’s reference implementation of SMI. The SMI is something to watch for but it remains pretty week thus far. Google’s platform has Anthos Service Mesh which is a commercial distribution based on Istio. AWS has its own AppMesh technology, also using Envoy proxy.
[1] Update from Apr 25, 2022 – Istio applied to become CNCF project.