The rise and rise of serverless
Written by Archit Rathi, investor at Oxx. Originally published in Startups Magazine.
Since AWS Lambda first launched in 2014, serverless computing has exploded. Adoption is high, and Amazon, Microsoft and Google all now offer serverless capabilities. As is so often the case, entrepreneurs have reacted fast, and created an ecosystem of startups that support the continuing growth of the serverless industry.
As a VC, I’m excited by these companies. Serverless hasn’t stopped growing yet, and nor has the accompanying ecosystem of supporting technologies.
But first: what is serverless, and how did we get here?
The main objective of serverless computing is to enable developers to build and run applications and services without thinking about servers. Servers still do exist, but the entire responsibility of infrastructure provisioning and management has now been outsourced to the cloud/service provider, with pricing on a “pay-per-compute-usage” basis. From ‘DevOps’ to ‘NoOps’ is the guiding vision/promise.
For entrepreneurs and tech startups, this minimises both a product’s time to market and opportunity costs. It’s huge.
Serverless first came onto the scene in 2014, with the launch of AWS Lambda, and now it is becoming ubiquitous. Microsoft and Google followed soon after Lambda with their own event-driven compute or ‘Functions as a Service’ platforms (Azure functions, Cloud Run), and now every cloud provider offers similar capabilities.
The business case for adopting serverless is simple: reduced costs, rapid pace of innovation or faster time-to-market, and automated infrastructure scaling. When speed is key, serverless rocks! About 50% of AWS users on DataDog have already adopted Lambda (Source).
So far, the pace of development has been led by the big tech companies building the bareboned infrastructure layer for serverless. Also, vendors such as Twilio, Salesforce and Atlassian have recently started providing serverless environments within their core offerings to drive growth and extend their developer ecosystems.
There is a downside though.
Serverless development also brings bottlenecks: a lack of admin control over underlying infrastructure and environment, cold application starts, and an even higher level of lock-in to your existing cloud providers.
So, why are frontier tech organizations still adopting serverless at a breakneck speed? Well, because all of these bottlenecks are temporary, and increasingly being mitigated by an emerging ecosystem of technology startups that are building the surrounding infrastructure layer.
This is the key question, and what I am interested in. So how is the surrounding ecosystem shaping up?
Knative, an open source project first developed by Google provides capabilities to run serverless applications on top of Kubernetes, helping you use the best of cloud-native stacks at one single place. Frameworks such as Serverless, Fission, Webiny, OpenFaas provide higher level constructs and help developers adjust to newer programming patterns.
As the number of functions and microservices in use increase over time, the monitoring/debugging activities also become quite complex. Startups such as Thundra, Dashbird, Lumigo, Epsagon now provide deep monitoring & observability capabilities across your entire cloud-native stack of serverless functions, microservices and containers, thereby also solving for lack of admin control / complex debugging issues.
No one can deny the criticality of securing user data, both from a customer trust perspective as well as from a legal perspective with new modern privacy regulations in place across the globe. As a consequence of using serverless architecture, engineers now build lots of loosely coupled services, thereby increasing the possible attack vector and herein lies the need for serverless security platforms. Cloud-native security startups such as Aqua Security and Snyk now cover this in their product portfolio and we have already seen a few acquisitions such as TwistLock, PureSec and Protego by major security vendors happen in the past year.
New use cases and workflows are being enabled by serverless!
As the infrastructure layer starts maturing, we are also in parallel seeing companies leveraging the concepts of serverless architecture to enable newer use cases. TriggerMesh, Koyeb, Vendia and Axiom are enabling new types of data management and multi-cloud workflows. Rockset is enabling real-time analytics at cloud scale. Anyscale lets developers build and develop distributed applications at scale. Evervault is building a data privacy/encryption platform for developers with a serverless function at its core. Ablyenables serverless integrations to build real-time applications and services.
I am enthused about the ecosystem as it stands, and I am eager to see what’s next. We’ve yet to see serverless being used in high performance computing environments such as for AI/ML/Big Data workloads in production. But I look forward to seeing use cases like these in action as the landscape matures going forward. We’re still in the early stages of this next evolution in cloud infrastructure, and I’m eager to see what founders are building for this new paradigm.