AWS Lambda and µServerless for Scala

Serverless… is a buzzword… However, it is here to stay.

I started using AWS lambda back in 2015. In those days adoption was still low, and not all companies were sure about the future of Serverless. Today, in 2018, serverless technologies continue to rise, and there is a huge amount of interest and adoption, tools are maturing, companies have entire production workloads running on serverless, and developers have an ever-growing number of resources at their disposal.

Despite its popularity, one common theme amongst many developers and the tools that they use to write lambda function is the lack of a structured model. Developers would create a script and then use tools like Serverless Framework to deploy it. In many instances, most of the lambda function code was copy-pasted from other functions. When you are rapid prototyping, this is all good, but once you move code into production it can be a hassle to deal with.

There are a few frameworks/middleware for python and JavaScript like python_decorators and middy that can help writing code for lambda, but a lot of the details for error handling, metrics, logging and so on are still left to the developer.

pronounced micro-serverless, is a framework for AWS lambda and Scala that hopes to provide:

1. Extensible error handling and notifications mechanism
2. Support for different types of configuration systems (env vars, SSM parameter store, noop)
3. Improve logging experience (structured logging coming soon!)
4. Custom handlers to easily deal with special events (SNS and API Gateway)
5. Simple and intuitive JSON de/serialization
6. Support for API Gateway and CORS configurations
7. Keep functions warm over time (if enabled)
8. Submit custom metrics to CloudWatch (coming soon!)
9. Provides a model to deploy general-purpose infrastructure* using the Serverless framework and LambdaSharpTool (coming soon!)

• General purpose infrastructure includes, but is not limited to, dead-letter queues, SNS topics, dynamo DB tables, and others.

µServerless is an open source project under the Apache2 license. See the GitHub and the Maven repositories for more information.

Getting started with µServerless

Add the following to your build.sbt:

libraryDependencies += "io.onema" %% "userverless-core" % "<LATEST_VERSION>"

When writing a µServerless function, you want to extend from the LambdaHandler base class. This is a generic class, so you have to tell it what is your event type and response type.

class Function extends LambdaHandler[S3Event, Unit] with NoopLambdaConfiguration

You also need to use one of the available configuration traits:

• EnvLambdaConfiguration (Environment variables)
• NoopLambdaConfiguration (Returns empty values, ideal for stubs)
• SsmLambdaConfiguration (Fetch values from SSM Parameter Store)

These traits proved a common interface to get configuration values from different sources, most of the time you will be using the Env config. The SSM configuration requires you to use the userverless-ssm-config package.

The entry point of your lambda function is ALWAYS the method lambdaHandler. When using the LambdaHandler class you have to implement the method execute.

The following is a simple function that copies objects from one s3 bucket to another:

import com.amazonaws.services.lambda.runtime.Context
import com.amazonaws.services.lambda.runtime.events.S3Event
import com.amazonaws.services.s3.{AmazonS3, AmazonS3ClientBuilder}
import com.amazonaws.services.s3.event.S3EventNotification.S3EventNotificationRecord
import io.onema.userverless.configuration.lambda.EnvLambdaConfiguration
import io.onema.userverless.function.LambdaHandler
import scala.collection.JavaConverters._


class CopyFunction extends LambdaHandler[S3Event, Unit] with EnvLambdaConfiguration {

  //--- Fields ---
  private val destinationBucket: String = getValue("destination/bucket").getOrElse("")
  private val s3Client: AmazonS3 = AmazonS3ClientBuilder.defaultClient()

  //--- Methods ---
  def execute(event: S3Event, context: Context): Unit = {
    event.getRecords.asScala.foreach(copy)
  }

  def copy(record: S3EventNotificationRecord): Unit = {
    val sourceBucket = record.getS3.getBucket.getName
    val sourceKey = record.getS3.getObject.getKey
    s3Client.copyObject(sourceBucket, sourceKey, destinationBucket, sourceKey)
  }
}

In this example, the getValue method returns an Option[String] with the value of the “DESTINATION_BUCKET” environment variable. If instead of using the EnvLambdaConfiguration trait we use the SsmLambdaConfiguration, it would fetch the value of the "/destination/bucket" parameter.

For convenience, there is an SnsHandler. This handler automatically decodes the message to the expected case class.

import com.amazonaws.services.lambda.runtime.Context
import io.onema.userverless.configuration.lambda.NoopLambdaConfiguration
import io.onema.userverless.function.SnsHandler

case class Foo(bar: String)

class Function extends SnsHandler[Foo] with NoopLambdaConfiguration {

  //--- Methods ---
  def execute(event: Foo, context: Context): Unit = {
    println(event.bar)
  }
}

Notice how in this case the expected type is Foo, the handler will unpack the SNS event and give the execute method the expected type.

When using API Gateway, we want to make sure that errors are handled gracefully and that the user of the API gets some information about what is going on in case of failure.

import com.amazonaws.serverless.proxy.model.{AwsProxyRequest, AwsProxyResponse}
import com.amazonaws.services.lambda.runtime.Context
import io.onema.userverless.configuration.lambda.NoopLambdaConfiguration
import io.onema.userverless.function.ApiGatewayHandler
import io.onema.userverless.function.ApiGatewayHandler.Cors
import org.apache.http.HttpStatus


class ApiGatewayFunction 
  extends ApiGatewayHandler 
  with Cors 
  with NoopLambdaConfiguration {

  //--- Methods ---
  override protected def corsConfiguration(origin: Option[String]) = new EnvCorsConfiguration(origin)
  
  def execute(request: AwsProxyRequest, context: Context): AwsProxyResponse = {
    cors(request) {
      new AwsProxyResponse(HttpStatus.SC_OK)
    }
  }
}

In this case, the function extends both the ApiGatewayHandler class and the Cors trait. When using CORS, you must create a configuration method that is used to get the authorized websites for the API.

For more information check out the documentation on GitHub.

I have created a couple of applications you can deploy using the serverless framework:

• LambdaMailer: Serverless application to send emails using SES and Lambda
• ServerlessLink: Serverless URL Shortener, take a look at the **demo site!

There is much work to do, and the vision for a framework that follows a model patterned after best practices is not fulfilled. I have a long list of things I still want to implement and I will continue to develop the framework including but not limited to:

• Adding support to deploy applications using the LambdaSharpTool
• Structure logging
• Better error reporting
• Metrics using CloudWatch
• Event listener middleware (currently implemented but not ideal)
• A tool to quickly create new applications
• A lot more, so stay tuned!