Files
swift-aws-lambda-runtime/Examples/Streaming+FunctionUrl/README.md
T
Sébastien Stormacq 3ddd64087d Add Streaming Lambda Examples with API Gateway and Function URL (#615)
## Overview

This PR reorganizes and enhances the streaming Lambda examples by
splitting them into two distinct examples that demonstrate different
invocation methods:

1. **Streaming+FunctionUrl** - Streaming responses via Lambda Function
URLs
2. **Streaming+APIGateway** - Streaming responses via API Gateway REST
API

## Changes

### 🔄 Restructured Examples

- **Renamed**: `Examples/Streaming/` → `Examples/Streaming+FunctionUrl/`
  - Maintains the original streaming example using Lambda Function URLs
  - Updated documentation to clarify Function URL-specific configuration
  - Improved AWS credentials handling in curl examples

- **New**: `Examples/Streaming+APIGateway/`
- Comprehensive example demonstrating API Gateway REST API with response
streaming
- Complete SAM template with proper IAM roles and streaming
configuration
  - Detailed documentation covering API Gateway-specific setup

### 📚 Documentation Improvements

#### Streaming+FunctionUrl
- Clarified that this example uses Lambda Function URLs
- Updated curl examples to use `eval $(aws configure export-credentials
--format env)` for cleaner credential handling
- Maintained all existing functionality and deployment instructions

#### Streaming+APIGateway (New)
- **316-line comprehensive README** covering:
  - Response streaming concepts and benefits
  - HTTP status code and header configuration
  - Streaming response body patterns
  - Local testing instructions
  - Complete SAM deployment guide with detailed template explanation
  - API Gateway-specific invocation with AWS Sigv4 authentication
  - Payload format documentation with example JSON
  - Security and reliability best practices
  - How API Gateway streaming works under the hood

### 🛠️ Technical Details

#### API Gateway Streaming Configuration
The new example demonstrates:
- Special Lambda URI: `/response-streaming-invocations` endpoint
- `responseTransferMode: STREAM` configuration
- IAM role with both `lambda:InvokeFunction` and
`lambda:InvokeWithResponseStream` permissions
- Proper timeout configuration (60s) to accommodate streaming duration

#### SAM Template Features
```yaml
- Lambda function with streaming support (arm64, provided.al2)
- API Gateway REST API with OpenAPI 3.0 definition
- IAM execution role for API Gateway to invoke Lambda with streaming
- Complete outputs for easy testing (API URL and Lambda ARN)
```

### 🔐 Security Enhancements

Both examples now include comprehensive security best practices:
- API Gateway access logging
- Throttling configuration
- AWS WAF integration recommendations
- Lambda concurrent execution limits
- Environment variable encryption
- Dead Letter Queue (DLQ) configuration
- VPC configuration guidance

### 🧪 Testing

Both examples support:
- **Local testing**: `swift run` with curl invocation on port 7000
- **AWS deployment**: Complete SAM templates with deployment
instructions
- **Authenticated invocation**: AWS Sigv4 examples with proper
credential handling

## Benefits

1. **Clearer separation**: Developers can now easily choose between
Function URLs and API Gateway based on their use case
2. **Better documentation**: Each example has tailored documentation for
its specific invocation method
3. **Production-ready**: Includes security best practices and proper IAM
configuration
4. **Easier testing**: Improved credential handling in curl examples

## Breaking Changes

None - this is purely additive. The original streaming example is
preserved as `Streaming+FunctionUrl`.

## Testing Checklist

- [x] Local testing works for both examples
- [x] SAM deployment templates are valid
- [x] Documentation is comprehensive and accurate
- [x] Security best practices are documented
- [x] Curl examples work with proper authentication

## Related Documentation

- [AWS Lambda Response
Streaming](https://docs.aws.amazon.com/lambda/latest/dg/configuration-response-streaming.html)
- [API Gateway Lambda Proxy Integration with
Streaming](https://docs.aws.amazon.com/apigateway/latest/developerguide/response-streaming-lambda-configure.html)
- [Lambda Function
URLs](https://docs.aws.amazon.com/lambda/latest/dg/lambda-urls.html)
EOF

---------

Co-authored-by: Sebastien Stormacq <stormacq@amazon.lu>
2025-12-05 16:38:50 -08:00

15 KiB
Raw Blame History

Streaming Lambda function

You can configure your Lambda function to stream response payloads back to clients. Response streaming can benefit latency sensitive applications by improving time to first byte (TTFB) performance. This is because you can send partial responses back to the client as they become available. Additionally, you can use response streaming to build functions that return larger payloads. Response stream payloads have a soft limit of 200 MB as compared to the 6 MB limit for buffered responses. Streaming a response also means that your function doesnt need to fit the entire response in memory. For very large responses, this can reduce the amount of memory you need to configure for your function.

Streaming responses incurs a cost. For more information, see AWS Lambda Pricing.

You can stream responses through Lambda function URLs, the AWS SDK, or using the Lambda InvokeWithResponseStream API. In this example, we create an authenticated Lambda function URL.

Code

The sample code creates a SendNumbersWithPause struct that conforms to the StreamingLambdaHandler protocol provided by the Swift AWS Lambda Runtime.

The handle(...) method of this protocol receives incoming events as a Swift NIO ByteBuffer and returns the output as a ByteBuffer.

The response is streamed through the LambdaResponseStreamWriter, which is passed as an argument in the handle function.

Setting HTTP Status Code and Headers

Before streaming the response body, you can set the HTTP status code and headers using the writeStatusAndHeaders(_:) method:

try await responseWriter.writeStatusAndHeaders(
    StreamingLambdaStatusAndHeadersResponse(
        statusCode: 200,
        headers: [
            "Content-Type": "text/plain",
            "x-my-custom-header": "streaming-example"
        ]
    )
)

The StreamingLambdaStatusAndHeadersResponse structure allows you to specify:

  • statusCode: HTTP status code (e.g., 200, 404, 500)
  • headers: Dictionary of single-value HTTP headers (optional)

Streaming the Response Body

After setting headers, you can stream the response body by calling the write(_:) function of the LambdaResponseStreamWriter with partial data repeatedly before finally closing the response stream by calling finish(). Developers can also choose to return the entire output and not stream the response by calling writeAndFinish(_:).

// Stream data in chunks
for i in 1...3 {
    try await responseWriter.write(ByteBuffer(string: "Number: \(i)\n"))
    try await Task.sleep(for: .milliseconds(1000))
}

// Close the response stream
try await responseWriter.finish()

An error is thrown if finish() is called multiple times or if it is called after having called writeAndFinish(_:).

Example Usage Patterns

The example includes two handler implementations:

  1. SendNumbersWithPause: Demonstrates basic streaming with headers, sending numbers with delays
  2. ConditionalStreamingHandler: Shows how to handle different response scenarios, including error responses with appropriate status codes

The handle(...) method is marked as mutating to allow handlers to be implemented with a struct.

Once the struct is created and the handle(...) method is defined, the sample code creates a LambdaRuntime struct and initializes it with the handler just created. Then, the code calls run() to start the interaction with the AWS Lambda control plane.

Build & Package

To build & archive the package, type the following commands.

swift package archive --allow-network-connections docker

If there is no error, there is a ZIP file ready to deploy. The ZIP file is located at .build/plugins/AWSLambdaPackager/outputs/AWSLambdaPackager/StreamingNumbers/StreamingNumbers.zip

Test locally

You can test the function locally before deploying:

swift run 

# In another terminal, test with curl:
curl -v --output response.txt \
  --header "Content-Type: application/json" \
  --data '"this is not used"' \
  http://127.0.0.1:7000/invoke

Deploy with the AWS CLI

Here is how to deploy using the aws command line.

Step 1: Create the function

# Replace with your AWS Account ID
AWS_ACCOUNT_ID=012345678901
aws lambda create-function \
--function-name StreamingNumbers \
--zip-file fileb://.build/plugins/AWSLambdaPackager/outputs/AWSLambdaPackager/StreamingNumbers/StreamingNumbers.zip \
--runtime provided.al2 \
--handler provided  \
--architectures arm64 \
--role arn:aws:iam::${AWS_ACCOUNT_ID}:role/lambda_basic_execution \
--timeout 15

Important

The timeout value must be bigger than the time it takes for your function to stream its output. Otherwise, the Lambda control plane will terminate the execution environment before your code has a chance to finish writing the stream. Here, the sample function stream responses during 3 seconds and we set the timeout for 5 seconds.

The --architectures flag is only required when you build the binary on an Apple Silicon machine (Apple M1 or more recent). It defaults to x64.

Be sure to set AWS_ACCOUNT_ID with your actual AWS account ID (for example: 012345678901).

Step2: Give permission to invoke that function through an URL

Anyone with a valid signature from your AWS account will have permission to invoke the function through its URL.

aws lambda add-permission \
  --function-name StreamingNumbers \
  --action lambda:InvokeFunctionUrl \
  --principal ${AWS_ACCOUNT_ID} \
  --function-url-auth-type AWS_IAM \
  --statement-id allowURL

Step3: Create the URL

This creates a URL with IAM authentication. Only calls with a valid signature will be authorized.

aws lambda create-function-url-config \
  --function-name StreamingNumbers \
  --auth-type AWS_IAM \
  --invoke-mode RESPONSE_STREAM 

This calls return various information, including the URL to invoke your function.

{
    "FunctionUrl": "https://ul3nf4dogmgyr7ffl5r5rs22640fwocc.lambda-url.us-east-1.on.aws/",
    "FunctionArn": "arn:aws:lambda:us-east-1:012345678901:function:StreamingNumbers",
    "AuthType": "AWS_IAM",
    "CreationTime": "2024-10-22T07:57:23.112599Z",
    "InvokeMode": "RESPONSE_STREAM"
}

Invoke your Lambda function

To invoke the Lambda function, use curl with the AWS Sigv4 option to generate the signature.

Read the AWS Credentials and Signature section for more details about the AWS Sigv4 protocol and how to obtain AWS credentials.

When you have the aws command line installed and configured, you will find the credentials in the ~/.aws/credentials file.

URL=https://ul3nf4dogmgyr7ffl5r5rs22640fwocc.lambda-url.us-east-1.on.aws/
REGION=us-east-1

# Set the AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY, and AWS_SESSION_TOKEN environment variables
eval $(aws configure export-credentials --format env)

curl "$URL"                              \
     --user "${AWS_ACCESS_KEY_ID}":"${AWS_SECRET_ACCESS_KEY}"   \
     --aws-sigv4 "aws:amz:${REGION}:lambda" \
     -H "x-amz-security-token: ${AWS_SESSION_TOKEN}" \
     --no-buffer

Note that there is no payload required for this example.

This should output the following result, with a one-second delay between each numbers.

1
2
3
Streaming complete!

Undeploy

When done testing, you can delete the Lambda function with this command.

aws lambda delete-function --function-name StreamingNumbers

Deploy with AWS SAM

Alternatively, you can use AWS SAM to deploy the Lambda function.

Prerequisites : Install the SAM CLI

SAM Template

The template file is provided as part of the example in the template.yaml file. It defines a Lambda function based on the binary ZIP file. It creates the function url with IAM authentication and sets the function timeout to 15 seconds.

AWSTemplateFormatVersion: '2010-09-09'
Transform: AWS::Serverless-2016-10-31
Description: SAM Template for StreamingLambda Example

Resources:
  # Lambda function
  StreamingNumbers:
    Type: AWS::Serverless::Function
    Properties:
      CodeUri: .build/plugins/AWSLambdaPackager/outputs/AWSLambdaPackager/StreamingNumbers/StreamingNumbers.zip
      Timeout: 15
      Handler: swift.bootstrap  # ignored by the Swift runtime
      Runtime: provided.al2
      MemorySize: 128
      Architectures:
        - arm64
      FunctionUrlConfig:
        AuthType: AWS_IAM
        InvokeMode: RESPONSE_STREAM

Outputs:
  # print Lambda function URL
  LambdaURL:
    Description: Lambda URL
    Value: !GetAtt StreamingNumbersUrl.FunctionUrl

Deploy with SAM

sam deploy \
--resolve-s3 \
--template-file template.yaml \
--stack-name StreamingNumbers \
--capabilities CAPABILITY_IAM 

The URL of the function is provided as part of the output.

CloudFormation outputs from deployed stack
-----------------------------------------------------------------------------------------------------------------------------
Outputs                                                                                                                                   
-----------------------------------------------------------------------------------------------------------------------------
Key                 LambdaURL                                                                                                             
Description         Lambda URL                                                                                                            
Value               https://gaudpin2zjqizfujfnqxstnv6u0czrfu.lambda-url.us-east-1.on.aws/                                                 
-----------------------------------------------------------------------------------------------------------------------------

Once the function is deployed, you can invoke it with curl, similarly to what you did when deploying with the AWS CLI.

# Set the AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY, and AWS_SESSION_TOKEN environment variables
eval $(aws configure export-credentials --format env)

curl "$URL"                              \
     --user "$AWS_ACCESS_KEY_ID":"$AWS_SECRET_ACCESS_KEY"   \
     --aws-sigv4 "aws:amz:${REGION}:lambda" \
     -H "x-amz-security-token: $AWS_SESSION_TOKEN" \
     --no-buffer

Undeploy with SAM

When done testing, you can delete the infrastructure with this command.

sam delete 

Payload decoding

The content of the input ByteBuffer depends on how you invoke the function:

Here is an example of Lambda function URL payload:

{
    "version": "2.0",
    "routeKey": "$default",
    "rawPath": "/",
    "rawQueryString": "",
    "headers": {
        "x-amzn-tls-cipher-suite": "TLS_AES_128_GCM_SHA256",
        "x-amzn-tls-version": "TLSv1.3",
        "x-amzn-trace-id": "Root=1-68762f44-4f6a87d1639e7fc356aa6f96",
        "x-amz-date": "20250715T103651Z",
        "x-forwarded-proto": "https",
        "host": "zvnsvhpx7u5gn3l3euimg4jjou0jvbfe.lambda-url.us-east-1.on.aws",
        "x-forwarded-port": "443",
        "x-forwarded-for": "2a01:cb0c:6de:8300:a1be:8004:e31a:b9f",
        "accept": "*/*",
        "user-agent": "curl/8.7.1"
    },
    "requestContext": {
        "accountId": "0123456789",
        "apiId": "zvnsvhpx7u5gn3l3euimg4jjou0jvbfe",
        "authorizer": {
            "iam": {
                "accessKey": "AKIA....",
                "accountId": "0123456789",
                "callerId": "AIDA...",
                "cognitoIdentity": null,
                "principalOrgId": "o-rlrup7z3ao",
                "userArn": "arn:aws:iam::0123456789:user/sst",
                "userId": "AIDA..."
            }
        },
        "domainName": "zvnsvhpx7u5gn3l3euimg4jjou0jvbfe.lambda-url.us-east-1.on.aws",
        "domainPrefix": "zvnsvhpx7u5gn3l3euimg4jjou0jvbfe",
        "http": {
            "method": "GET",
            "path": "/",
            "protocol": "HTTP/1.1",
            "sourceIp": "2a01:...:b9f",
            "userAgent": "curl/8.7.1"
        },
        "requestId": "f942509a-283f-4c4f-94f8-0d4ccc4a00f8",
        "routeKey": "$default",
        "stage": "$default",
        "time": "15/Jul/2025:10:36:52 +0000",
        "timeEpoch": 1752575812081
    },
    "isBase64Encoded": false
}

⚠️ Security and Reliability Notice

These are example applications for demonstration purposes. When deploying such infrastructure in production environments, we strongly encourage you to follow these best practices for improved security and resiliency: