An API (Application Programming Interface) is the visible part of a microservice. A microservice architecture is a modern variant of the service-oriented architecture to facilitate both the development and the operations of complex IT systems. In a microservices architecture, an application is arranged as a collection of loosely coupled micro-level services that are built around business capabilities. These microservices can be developed and deployed independently of each other and even different versions can co-exist in operations. The protocols to couple the microservices are lightweight, often an HTTP resource API.
A microservice architecture has the promise to deliver:
- Scalability
- Availability & resiliency & failure isolation
- Independency & autonomy
- Decentralised governance
- Continuous delivery through DevOps
To make those microservices enterprise-grade, you will need more control over the infrastructure, build process, continued performance, security, resilience, etc. to effectively scale globally.
Cloud Containers and Cloud Functions are two techniques to create an isolated execution environment where a microservice can run, it has all the necessary components to run inside but it follows a standard and it clearly defines its requirements and dependencies. Cloud Containers and Cloud Functions allow the DevOps team to manage the microservices life cycle independently of the language and platform they use, and independently of the environment. This not only scales well during development, it also scales well in operations. And because no servers need to be stopped when deploying a new release, they truly support zero-downtime and build-in resilience.
You can think of microservices as the Dev in DevOps, and Cloud Functions or Cloud Containers as the Ops. Cloud Functions and Cloud Containers are often viewed as competing technologies. But when integrated, they can be a powerful combination. This post describes the pros and cons of both and how they can be combined.
For the Dev in DevOps, please see our post on How to industrialise API development.
What is a Cloud Function?
Function-as-a-service (FaaS) was introduced in 2010 and picked by Amazon in 2014, dubbed AWS Lambda. Later other public cloud vendor followed, e.g. Google Cloud Functions and Microsoft Azure Functions.
Function-as-a-service or Cloud Functions are often referred to as “serverless” even though this is a misnomer since servers are certainly involved. Cloud Functions basically replace long-running virtual machines with ephemeral “compute power” that comes into existence on request and disappears immediately after use.
With Cloud Functions, you configure events, such as API requests or file uploads, that trigger your serverless function to execute. And when that action is complete, the server goes idle until another action is requested. And when multiple events trigger the function at the same time, multiple instances of the serverless function are automatically launched, without coding effort from your developers.
Cloud Functions truly support the development model of event-driven, stateless microservices.
Pros of Cloud Functions
The first pro is that you only pay for time when the server is executing the action. Like mentioned earlier, the server only runs when an event triggers it to, so you’ll only pay for the short time when the server is active. Save some bucks!
Additionally, serverless allows your app to be elastic. It automatically scales out to accommodate many concurrent requests and scales back down when traffic subsides. This characteristic increases the performance of your app while saving you money.
You’ll also spend much less time managing servers. You don’t need to worry about provisioning infrastructure, managing capacity, or avoiding downtime – your cloud service provider does all that.
As such, serverless architectures help you reduce development time and get your products to market faster. If you don’t need to focus on your infrastructure, you can spend your time on building the best product possible. This is one of the key benefits that Jack and his team saw in serverless.
Serverless also fits really well with microservices, where developers build smaller, more loosely-coupled parts of the software whole. Serverless alleviates the need for these developers to spin up their own server instances and allows them to build these microservices much more quickly.
And because serverless functions don’t need a long-term hosting location, you don’t need to assign them to specific cloud servers and aren’t limited to particular availability zones. This essentially makes them highly available.
Cons of Cloud Functions
Wake-up time is a concern because Cloud Functions sit cold until it is triggered by an application. Thus, serverless may not be an ideal solution for applications where speed is paramount. You can use a work-around and keep the Cloud Functions warm.
Vendor lock-in is certainly a concern. If you use the serverless offering of your cloud service provider (“cloud vendor”) and you decide to migrate to another cloud vendor, you’ll likely have to make major changes to your code base. And this can take a lot of time and money. This was a major drawback in the eyes of Jack and his team, as vendor lock-in was a primary concern.
To deal with this, you can use frameworks such as the Serverless Framework or Spring Cloud Function, which abstract serverless away from the cloud vendor and allow you to jump from one provider to another more easily. But you’ll certainly lose some functionality along the way.
Due to the event-based nature of Cloud Functions, it’s not the best choice for long-running apps. Online games and apps that perform analysis of very large datasets won’t be a good fit for a serverless architecture, as serverless functions have time limits (typically five minutes) before they are terminated.
Another con of Cloud Functions is that you don’t have much control over the server (that’s why they are called serverless). Usually you can select the amount of memory your function gets, but then your cloud vendor assigns a small amount of disk storage and decides the rest of the specifications for you. This can be a hindrance if you need something like a GPU to process large image or video files.
Cloud Functions also do not support all programming languages. AWS Lambda functions for example support Java, Go, PowerShell, Node. js, C#, Python, and Ruby code but do not support PHP.
Finally, complex apps can be hard to build using a serverless architecture. You’ll have to perform a lot of coordination and manage dependencies between all of the serverless functions, which can be a tough job for large, complicated applications.
What are Cloud Containers?
Containerisation is developed as an alternative to “virtualisation” known from the early days of cloud computing (infrastructure-as-a-service, IaaS). It involves encapsulating or packaging up software code and all its dependencies so that it can run uniformly and consistently on any infrastructure.
With traditional development methods running on your own servers or on virtualised servers, code is developed in a specific computing environment which, when transferred to a new location, often results in bugs and errors. Containerisation eliminates this problem by bundling the application code together with the related configuration files, libraries, and dependencies required for it to run. This single package of software or “container” is abstracted away from the host operating system, and hence, it stands alone and becomes portable—able to run across any platform or cloud, free of issues.
The concept of containerisation and process isolation is decades old, but the emergence of the open source Docker Engine in 2013, an industry standard for containers with simple developer tools and a universal packaging approach, accelerated the adoption of this technology.
Cloud Containers are often referred to as “lightweight,” meaning they share the machine’s operating system kernel and do not require the overhead of associating an operating system within each application. Cloud Containers are inherently smaller in capacity than a VM and require less start-up time, allowing far more containers to run on the same compute capacity as a single VM. This drives higher server efficiencies and, in turn, reduces server and licensing costs. Cloud Containers, however, are significantly bigger than Cloud Functions.
Pros of Cloud Containers
Portability is a clear benefit of containers. The main draw of a container is that you can combine the application and all of its dependencies into a neat little package and run it anywhere. This provides an unprecedented level of flexibility and portability, and allows you to stay cloud vendor-agnostic.
The next pro, especially compared to serverless architectures, is that you have full control over your application.
You are the master of your domain. You can lord over individual containers, the entire container ecosystem, and the servers they run on. You can manage all the resources, set all of the policies, oversee security, and determine how your application is deployed and behaves. You can debug and monitor as you please. This isn’t the case for serverless; rather, all of that is managed by your cloud vendor.
Container-based applications can also be as large and complex as you need them to be, as there are no memory or time limitations like there are with serverless.
Cons of Cloud Containers
The first con is that Cloud Containers take much more work to set up and manage than Cloud Functions. Really much more.
Not only during the initial set-up. Every time you make a change to your codebase, you’ll need to package the container and ensure all of the containers communicate with each other properly before deploying into production. You’ll also need to keep containers’ operating systems up to date with frequent security fixes and other patches. And you have to figure out which containers go on which servers. All of this can slow down your development process.
Because containers need a long-running hosting location, they are more expensive to run than serverless functions. With serverless, you only pay when servers execute your function. With containers, you have to pay for server usage even if they’re sitting idle.
Cloud Containers face some scaling challenges as well.
The first issue is with monitoring. As an application grows, more and more containers are added. And these containers are highly dispersed, scattered, and constantly changing, thus making monitoring a nightmare.
It’s also difficult for data and storage to scale with the increasing number of containers.
First, there is no data persistence when containers are rescheduled or destroyed, so data is often lost when changes are made. Next, with the distributed nature of containers, it’s difficult to move data between different locations or cloud service providers. Storage also doesn’t scale well with apps, which lead to unpredictable performance issues.
To run containers at scale, you need an orchestration mechanism for automating application deployment, up and down scaling, monitoring and management. One of the most popular open-source container-orchestration system is Kubernetes (commonly stylized as k8s). It was originally designed by Google and is now maintained by the Cloud Native Computing Foundation. It aims to provide a platform for automating deployment, scaling, and operations of application containers across clusters of hosts. It works with a range of container tools and runs containers in a cluster, often with images built using Docker.
Many cloud services offer a Kubernetes-based platform-as-a-service (PaaS) on which Kubernetes can be deployed, e.g. Microsoft Azure Kubernetes Service (AKS) and Amazon Elastic Container Service for Kubernetes (EKS).
Kubernetes defines a set of building blocks (“primitives”), which collectively provide mechanisms that deploy, maintain, and scale applications based on CPU, memory or custom metrics. Kubernetes is loosely coupled and extensible to meet different workloads. This extensibility is provided in large part by the Kubernetes API, which is used by internal components as well as extensions and containers that run on Kubernetes. The platform exerts its control over compute and storage resources by defining resources as Objects, which can then be managed as such.
Kubernetes follows the primary/replica architecture. The components of Kubernetes can be divided into those that manage an individual node and those that are part of the control plane.
When should you use each?
Cloud Containers are best used for complex, long-running applications where you need a high level of control over your environment and you have the resources to set up and maintain the application.
Cloud Containers are also very useful in migrating monolithic legacy applications to the cloud. You can split up these apps into containerized microservices and orchestrate them using a tool like Kubernetes or Docker Swarm.
Cloud Containers are ideal for an app like a large e-commerce website. A site like this contains (pun intended) many parts, such as product listings, payment processing, inventory management, and many more. You can use containers to package each of these services without having to worry about time limits or memory issues.
Cloud Functions are best applied for apps that need to be ready to perform tasks but don’t always need to be running. For instance, serverless is a great choice for an Internet of Things (IoT) application that detects the presence of water to identify a leak in a water storage facility. The app doesn’t have to run all the time, but it needs to be ready to act in the case of a leak.
Cloud Functions are ideal when development speed and cost minimization is paramount and if you don’t want to manage scaling concerns.
How can you use both?
Cloud Functions and Cloud Containers have strengths that can compliment the other’s weaknesses, and incorporating these two technologies together can be highly beneficial.
You can build a large, complex application with a microservices using Cloud Containers. But the application can hand off some back-end tasks, such as data transfer, file backups, and alert triggering, to Cloud Functions. This would save money and could increase the application’s performance.
Amazon Fargate is a tool that is sort of a hybrid between Cloud Functions and Cloud Containers that can help alleviate some of the issues that each of these technologies presents. Fargate is a compute engine for Amazon Elastic Container Service (ECS) and Amazon Elastic Container Service for Kubernetes (EKS) that lets you run containers without having to manage servers or clusters. You don’t have to provision, configure, or scale virtual servers to run containers. Thus, Fargate combines the portability of containers with the elasticity and ease of use of serverless.
At its core, Fargate is a container service (with serverless attributes), so portability isn’t a concern. Fargate also works with Kubernetes via EKS and thus is very portable to other cloud platforms such as Azure (Azure Kubernetes Service) or Google Cloud Platform (Google Kubernetes Engine).
Additional concerns with serverless include the five-minute maximum runtime and memory limitations. In order to get around this, Fargate can be used to create a “Fat Lambda”. To create a Fat Lambda, you:
- Wrap your code in a container
- Deploy that container to Elastic Container Repository (ECR)
- Run that container in a task in Fargate (which can either be scheduled with CloudWatch or triggered by a Lambda)
- Eventually convert this to run in an ECS EC2 cluster, if necessary
- When a task gets too big for Lambda, you can trigger the Fat Lambda to run.
An issue with containers is the aforementioned scalability problem. Fargate allows you to scale without managing servers, which allows developers to focus on building product.
There’s a lot of activity around combining serverless and containers because there are so many benefits that can come out of this powerful marriage.
Controlled AI services 