Files
swift-aws-lambda-runtime/Examples/StreamingFromEvent
Sébastien Stormacq 412a345bdd [core] Add user-facing API for Streaming Lambda functions that receive JSON events (#532)
Add user-facing API for Streaming Lambda functions that receives JSON
events

### Motivation:

Streaming Lambda functions developed by developers had no choice but to
implement a handler that receives incoming data as a `ByteBuffer`. While
this is useful for low-level development, I assume most developers will
want to receive a JSON event to trigger their streaming Lambda function.

Going efficiently from a `ByteBuffer` to a Swift struct requires some
code implemented in the `JSON+ByteBuffer.swift` file of the librray. We
propose to further help developers by providing them with a new
`handler()` function that directly receives their `Decodable` type.

### Modifications:

This PR adds a public facing API (+ unit test + updated README) allowing
developers to write a handler method accepting any `Decodable` struct as
input.

```swift
import AWSLambdaRuntime
import NIOCore

// Define your input event structure
struct StreamingRequest: Decodable {
    let count: Int
    let message: String
    let delayMs: Int?
}

// Use the new streaming handler with JSON decoding
let runtime = LambdaRuntime { (event: StreamingRequest, responseWriter, context: LambdaContext) in
    context.logger.info("Received request to send \(event.count) messages")
    
    // Stream the messages
    for i in 1...event.count {
        let response = "Message \(i)/\(event.count): \(event.message)\n"
        try await responseWriter.write(ByteBuffer(string: response))
        
        // Optional delay between messages
        if let delay = event.delayMs, delay > 0 {
            try await Task.sleep(for: .milliseconds(delay))
        }
    }
    
    // Finish the stream
    try await responseWriter.finish()
    
    // Optional: Execute background work after response is sent
    context.logger.info("Background work: processing completed")
}

try await runtime.run()
```

This interface provides:
- **Type-safe JSON input**: Automatic decoding of JSON events into Swift
structs
- **Streaming responses**: Full control over when and how to stream data
back to clients
- **Background work support**: Ability to execute code after the
response stream is finished
- **Familiar API**: Uses the same closure-based pattern as regular
Lambda handlers

Because streaming Lambda functions can be invoked either directly
through the API or through Lambda Function URL, this PR adds the
decoding logic to support both types, shielding developers from working
with Function URL requests and base64 encoding.

We understand these choice will have an impact on the raw performance
for event handling. Those advanced users that want to get the maximum
might use the existing `handler(_ event: ByteBuffer, writer:
LambaStreamingWriter)` function to implement their own custom decoding
logic.

This PR provides a balance between ease of use for 80% of the users vs
ultimate performance, without closing the door for the 20% who need it.

### Result:

Lambda function developers can now use arbitrary `Decodable` Swift
struct or Lambda events to trigger their streaming functions. 🎉

---------

Co-authored-by: Tim Condon <0xTim@users.noreply.github.com>
2025-07-23 19:03:25 +04:00
..

Streaming Codable Lambda function

This example demonstrates how to use the StreamingLambdaHandlerWithEvent protocol to create Lambda functions that:

  1. Receive JSON input: Automatically decode JSON events into Swift structs
  2. Stream responses: Send data incrementally as it becomes available
  3. Execute background work: Perform additional processing after the response is sent

The example uses the streaming codable interface that combines the benefits of:

  • Type-safe JSON input decoding (like regular LambdaHandler)
  • Response streaming capabilities (like StreamingLambdaHandler)
  • Background work execution after response completion

Streaming responses incurs a cost. For more information, see AWS Lambda Pricing.

You can stream responses through Lambda function URLs, the AWS SDK, or using the Lambda InvokeWithResponseStream API.

Code

The sample code creates a StreamingFromEventHandler struct that conforms to the StreamingLambdaHandlerWithEvent protocol provided by the Swift AWS Lambda Runtime.

The handle(...) method of this protocol receives incoming events as a decoded Swift struct (StreamingRequest) and returns the output through a LambdaResponseStreamWriter.

The Lambda function expects a JSON payload with the following structure:

{
  "count": 5,
  "message": "Hello from streaming Lambda!",
  "delayMs": 1000
}

Where:

  • count: Number of messages to stream (1-100)
  • message: The message content to repeat
  • delayMs: Optional delay between messages in milliseconds (defaults to 500ms)

The response is streamed through the LambdaResponseStreamWriter, which is passed as an argument in the handle function. The code calls the write(_:) function of the LambdaResponseStreamWriter with partial data written repeatedly before finally closing the response stream by calling finish(). Developers can also choose to return the entire output and not stream the response by calling writeAndFinish(_:).

An error is thrown if finish() is called multiple times or if it is called after having called writeAndFinish(_:).

The handle(...) method is marked as mutating to allow handlers to be implemented with a struct.

Once the struct is created and the handle(...) method is defined, the sample code creates a LambdaRuntime struct and initializes it with the handler just created. Then, the code calls run() to start the interaction with the AWS Lambda control plane.

Key features demonstrated:

  • JSON Input Decoding: The function automatically parses the JSON input into a StreamingRequest struct
  • Input Validation: Validates the count parameter and returns an error message if invalid
  • Progressive Streaming: Sends messages one by one with configurable delays
  • Timestamped Output: Each message includes an ISO8601 timestamp
  • Background Processing: Performs cleanup and logging after the response is complete
  • Error Handling: Gracefully handles invalid input with descriptive error messages

Build & Package

To build & archive the package, type the following commands.

swift package archive --allow-network-connections docker

If there is no error, there is a ZIP file ready to deploy. The ZIP file is located at .build/plugins/AWSLambdaPackager/outputs/AWSLambdaPackager/StreamingFromEvent/StreamingFromEvent.zip

Test locally

You can test the function locally before deploying:

swift run 

# In another terminal, test with curl:
curl -v \
  --header "Content-Type: application/json" \
  --data '{"count": 3, "message": "Hello World!", "delayMs": 1000}' \
  http://127.0.0.1:7000/invoke

Or simulate a call from a Lambda Function URL (where the body is encapsulated in a Lambda Function URL request):

curl -v \
  --header "Content-Type: application/json" \
  --data @events/sample-request.json \
  http://127.0.0.1:7000/invoke

Deploy with the AWS CLI

Here is how to deploy using the aws command line.

Step 1: Create the function

# Replace with your AWS Account ID
AWS_ACCOUNT_ID=012345678901
aws lambda create-function \
--function-name StreamingFromEvent \
--zip-file fileb://.build/plugins/AWSLambdaPackager/outputs/AWSLambdaPackager/StreamingFromEvent/StreamingFromEvent.zip \
--runtime provided.al2 \
--handler provided \
--architectures arm64 \
--role arn:aws:iam::${AWS_ACCOUNT_ID}:role/lambda_basic_execution

Important

The timeout value must be bigger than the time it takes for your function to stream its output. Otherwise, the Lambda control plane will terminate the execution environment before your code has a chance to finish writing the stream. Here, the sample function stream responses during 10 seconds and we set the timeout for 15 seconds.

The --architectures flag is only required when you build the binary on an Apple Silicon machine (Apple M1 or more recent). It defaults to x64.

Be sure to set AWS_ACCOUNT_ID with your actual AWS account ID (for example: 012345678901).

Step 2: Give permission to invoke that function through a URL

Anyone with a valid signature from your AWS account will have permission to invoke the function through its URL.

aws lambda add-permission \
  --function-name StreamingFromEvent \
  --action lambda:InvokeFunctionUrl \
  --principal ${AWS_ACCOUNT_ID} \
  --function-url-auth-type AWS_IAM \
  --statement-id allowURL

Step 3: Create the URL

This creates a URL with IAM authentication. Only calls with a valid signature will be authorized.

aws lambda create-function-url-config \
  --function-name StreamingFromEvent \
  --auth-type AWS_IAM \
  --invoke-mode RESPONSE_STREAM 

This call returns various information, including the URL to invoke your function.

{
    "FunctionUrl": "https://ul3nf4dogmgyr7ffl5r5rs22640fwocc.lambda-url.us-east-1.on.aws/",
    "FunctionArn": "arn:aws:lambda:us-east-1:012345678901:function:StreamingFromEvent",
    "AuthType": "AWS_IAM",
    "CreationTime": "2024-10-22T07:57:23.112599Z",
    "InvokeMode": "RESPONSE_STREAM"
}

Invoke your Lambda function

To invoke the Lambda function, use curl with the AWS Sigv4 option to generate the signature.

Read the AWS Credentials and Signature section for more details about the AWS Sigv4 protocol and how to obtain AWS credentials.

When you have the aws command line installed and configured, you will find the credentials in the ~/.aws/credentials file.

URL=https://ul3nf4dogmgyr7ffl5r5rs22640fwocc.lambda-url.us-east-1.on.aws/
REGION=us-east-1
ACCESS_KEY=AK...
SECRET_KEY=...
AWS_SESSION_TOKEN=...

curl --user "${ACCESS_KEY}":"${SECRET_KEY}"   \
     --aws-sigv4 "aws:amz:${REGION}:lambda" \
     -H "x-amz-security-token: ${AWS_SESSION_TOKEN}" \
     --no-buffer \
     --header "Content-Type: application/json" \
     --data '{"count": 3, "message": "Hello World!", "delayMs": 1000}' \
     "$URL"  

This should output the following result, with configurable delays between each message:

[2024-07-15T05:00:00Z] Message 1/3: Hello World!
[2024-07-15T05:00:01Z] Message 2/3: Hello World!
[2024-07-15T05:00:02Z] Message 3/3: Hello World!
✅ Successfully sent 3 messages

Undeploy

When done testing, you can delete the Lambda function with this command.

aws lambda delete-function --function-name StreamingFromEvent

Deploy with AWS SAM

Alternatively, you can use AWS SAM to deploy the Lambda function.

Prerequisites : Install the SAM CLI

SAM Template

The template file is provided as part of the example in the template.yaml file. It defines a Lambda function based on the binary ZIP file. It creates the function url with IAM authentication and sets the function timeout to 15 seconds.

AWSTemplateFormatVersion: '2010-09-09'
Transform: AWS::Serverless-2016-10-31
Description: SAM Template for StreamingFromEvent Example

Resources:
  # Lambda function
  StreamingNumbers:
    Type: AWS::Serverless::Function
    Properties:
      CodeUri: .build/plugins/AWSLambdaPackager/outputs/AWSLambdaPackager/StreamingFromEvent/StreamingFromEvent.zip
      Timeout: 15
      Handler: swift.bootstrap  # ignored by the Swift runtime
      Runtime: provided.al2
      MemorySize: 128
      Architectures:
        - arm64
      FunctionUrlConfig:
        AuthType: AWS_IAM
        InvokeMode: RESPONSE_STREAM

Outputs:
  # print Lambda function URL
  LambdaURL:
    Description: Lambda URL
    Value: !GetAtt StreamingNumbersUrl.FunctionUrl

Deploy with SAM

sam deploy \
--resolve-s3 \
--template-file template.yaml \
--stack-name StreamingFromEvent \
--capabilities CAPABILITY_IAM 

The URL of the function is provided as part of the output.

CloudFormation outputs from deployed stack
-----------------------------------------------------------------------------------------------------------------------------
Outputs                                                                                                                                   
-----------------------------------------------------------------------------------------------------------------------------
Key                 LambdaURL                                                                                                             
Description         Lambda URL                                                                                                            
Value               https://gaudpin2zjqizfujfnqxstnv6u0czrfu.lambda-url.us-east-1.on.aws/                                                 
-----------------------------------------------------------------------------------------------------------------------------

Once the function is deployed, you can invoke it with curl, similarly to what you did when deploying with the AWS CLI.

curl -X POST \
     --data '{"count": 3, "message": "Hello World!", "delayMs": 1000}' \
     --user "$ACCESS_KEY":"$SECRET_KEY"   \
     --aws-sigv4 "aws:amz:${REGION}:lambda" \
     -H "x-amz-security-token: $AWS_SESSION_TOKEN" \
     --no-buffer \
     "$URL"

Undeploy with SAM

When done testing, you can delete the infrastructure with this command.

sam delete