Serverless applications built with AWS Lambda offer agility, scalability, and efficiency, but they lack built-in persistent storage. When Lambda functions need to access files across invocations or share data persistently, Cloud Native Qumulo (CNQ) bridges this gap.

This blog demonstrates how to connect AWS Lambda to CNQ via NFS, enabling persistent and scalable file access for your serverless applications.

In this blog:

• What Is AWS Lambda?
• Why AWS Lambda Needs Persistent Storage
• Setting Up CNQ NFS Access for Lambda
• Creating a Lambda Function That Interacts with CNQ
• Packaging and Deploying the Lambda
• Putting It All to the Test

What Is AWS Lambda?

AWS Lambda is a serverless, event-driven compute service that runs your code in response to events, such as HTTP requests, file uploads, or data stream changes, without the need to provision or manage servers. It handles scaling, maintenance, and logging, so you can focus on writing code in your language of choice. Functions run only when triggered, and you’re billed only for the compute time used.

Lambda is particularly well-suited for scenarios requiring rapid scaling and event-driven processing:

• File Processing: Trigger workflows in real time upon file uploads to Amazon S3.
• Stream Processing: Analyze and process real-time streaming data from sources like Amazon Kinesis.
• Web Applications: Build scalable web apps by integrating Lambda with other AWS services.
• IoT Backends: Manage backend operations for IoT devices without server management.

⚠️ However, Lambda is stateless by design. Functions have no built-in persistent storage, making external solutions necessary for file persistence.

Why AWS Lambda Needs Persistent Storage

Serverless computing has changed application development by enabling developers to build highly scalable functions without the need for complicated server infrastructure. While Lambda functions natively interact with object storage, such as Amazon S3, many real-world scenarios require accessing traditional file systems, whether to support legacy applications or to leverage the advantages that file-based storage offers.

Amazon’s built-in persistent storage option, Elastic File System (EFS) lacks SMB protocol support, limiting its use to legacy workflows that rely on NFS. By integrating AWS Lambda with Cloud Native Qumulo (CNQ), developers can bridge this gap seamlessly. CNQ’s multiprotocol capabilities enable serverless applications to directly access legacy file systems without compromising scalability or simplicity.

With CNQ and AWS Lambda, stateful workflows become fully serverless, delivering unmatched speed, ease-of-use, and robust storage power – all without complexity.

Many real-world applications need persistent file storage:

• Generating and saving reports
• Processing uploaded files
• Sharing data across invocations or services

Lambda’s temporary /tmp storage is ephemeral. CNQ offers persistent, high-performance, multi-protocol, and scalable NFS storage, making it ideal for these scenarios.

Setting Up CNQ NFS Access for Lambda

Deploy CNQ in Your AWS VPC: Deploy CNQ via the Qumulo Web UI or CLI in minutes.

• Cloud Native Qumulo on AWS Administrator Guide
• How Cloud Native Qumulo on AWS Works
• Deploying Cloud Native Qumulo on AWS with CloudFormation
• Deploying Cloud Native Qumulo on AWS with Terraform

Create a Qumulo NFS Export:

Use the UI, API, or the qq CLI to create an export for your Lambda use case.

Ensure your NFS export:

• Has read/write permissions
• Is accessible within the same VPC/subnet as Lambda
• Allows non-root port connections (Lambda does not run as root)

Configure CNQ Security:

Ensure ingress access on port 2049/tcp (NFS):

Add NFS to the VPC Security Group ingress rules, replacing the CIDR below with your Lambda subnet’s actual CIDR range.

aws ec2 authorize-security-group-ingress \
  --group-id sg-0a1b2c3d4e5f6g7h8 \
  --protocol tcp \
  --port 2049 \
  --cidr 192.168.0.0/24

Creating the Lambda Function Code

Create a Python handler file (for example, test_cnq_access.py) with the following code.
Note: Replace YOUR_CNQ_ADDRESS with your CNQ cluster’s DNS name (or IP address) and YOUR_NFS_EXPORT with your actual NFS export path (e.g., lambda-data).

import os
import sys
from datetime import datetime
import libnfs

os.environ['LD_LIBRARY_PATH'] = "/var/task/lib:/var/task/libnfs:" + os.environ.get('LD_LIBRARY_PATH', '')
sys.path.append('/var/task')

def handler(event, context):
    try:
        nfs = libnfs.NFS("nfs://YOUR_CNQ_ADDRESS/YOUR_NFS_EXPORT")
    except Exception as e:
        return {"statusCode": 500, "body": "Error connecting to NFS export: " + str(e)}
    
    try:
        timestamp = datetime.now().isoformat()
        message = f"Hello from AWS Lambda via CNQ! - {timestamp}\n"
        
        f = nfs.open("/testing-from-lambda.txt", "w+")
        f.write(message)
        f.close()
    except Exception as e:
        return {"statusCode": 500, "body": "Error writing to file: " + str(e)}
    
    try:
        f = nfs.open("/testing-from-lambda.txt", "r")
        content = f.read()
        return {"statusCode": 200, "body": content}
    except Exception as e:
        return {"statusCode": 500, "body": "Error reading file: " + str(e)}

Packaging the Lambda Function

Since AWS Lambda cannot mount FUSE or use kernel-level NFS mounts, we package the native libnfs shared library along with the Python code to use a user-space alternative.

IMPORTANT: Build and test in an Amazon Linux 2 environment for compatibility.

Deployment Package Creation Script – Create a shell script (or run these commands step by step). Adjust as needed for your environment.

#!/bin/bash
################################################################
# AWS Lambda Deployment Package Creation Script
# Run these steps on an Amazon Linux 2 VM or container.
################################################################

BUILD_DIR="/home/ec2-user/build"

echo "Creating package directories..."
mkdir -p "$BUILD_DIR/package/lib"

echo "Installing prerequisite packages..."
sudo yum -y install python3.8 python38-devel gcc libnfs-devel
sudo amazon-linux-extras enable python3.8

echo "Installing libnfs using pip3.8..."
pip3.8 install libnfs --target ./package

echo "Copying AL2 native libnfs shared library into the package..."
cp -L /usr/lib64/libnfs.so package/lib/

echo "Creating a libnfs symlink in package (needed at runtime)..."
cd package/lib 
ln -sf libnfs.so libnfs.so.8

cd $BUILD_DIR

echo "Copying test_cnq_access.py Lambda handler script..."
cp test_cnq_access.py package/

echo "Zipping deployment package (function.zip)..."
cd package
zip -r ../function.zip .

IAM Configuration

Create a trust policy(trust-policy.json):

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Principal": {
        "Service": "lambda.amazonaws.com"
      },
      "Action": "sts:AssumeRole"
    }
  ]
}

Create and attach required policies:

aws iam create-role \
  --role-name LambdaCNQExecutionRole \
  --assume-role-policy-document file://trust-policy.json \
  --description "Lambda execution role for CNQ functions"

aws iam attach-role-policy \
  --role-name LambdaCNQExecutionRole \
  --policy-arn arn:aws:iam::aws:policy/service-role/AWSLambdaBasicExecutionRole

aws iam attach-role-policy \
  --role-name LambdaCNQExecutionRole \
  --policy-arn arn:aws:iam::aws:policy/service-role/AWSLambdaVPCAccessExecutionRole

Creating the Lambda Function

Replace “123456789012” with your AWS Account ID.

aws lambda create-function \
  --function-name cnq-nfs-lambda \
  --runtime python3.8 \
  --role arn:aws:iam::123456789012:role/LambdaCNQExecutionRole \
  --handler test_cnq_access.handler \
  --zip-file fileb://function.zip

Configuring VPC Settings

To allow your Lambda function to access resources in a VPC, update its configuration with your subnet and security group details.

aws lambda update-function-configuration \
  --function-name cnq-nfs-lambda \
  --vpc-config '{
    "SubnetIds": ["subnet-0a1b2c3d", "subnet-4e5f6g7h"],
    "SecurityGroupIds": ["sg-0abc123def456ghi7"]
  }'

Optional: Attaching Custom Policies Tailored to Your Use Case

If your Lambda function needs to access additional services (like S3), create a custom policy. For example:

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Sid": "AllowCloudWatchLogs",
      "Effect": "Allow",
      "Action": [
        "logs:CreateLogGroup",
        "logs:CreateLogStream",
        "logs:PutLogEvents"
      ],
      "Resource": "arn:aws:logs:*:*:*"
    },
    {
      "Sid": "AllowS3Access",
      "Effect": "Allow",
      "Action": [
        "s3:GetObject",
        "s3:PutObject",
        "s3:ListBucket"
      ],
      "Resource": [
        "arn:aws:s3:::YOUR_BUCKET_NAME",
        "arn:aws:s3:::YOUR_BUCKET_NAME/*"
      ]
    }
  ]
}

Note: Be sure to use the least-privilege principle in production.

Putting It to the Test

With everything set up, it’s time to see your work in action:
Trigger the Lambda Function:

aws lambda invoke \
  --function-name cnq-nfs-lambda \
  --payload '{}' \
  --cli-binary-format raw-in-base64-out \
  response.json

Verify the function by mounting the CNQ NFS export on a test system and viewing the file:

$ sudo mkdir /mnt/test
$ sudo mount -t nfs -o vers=3,tcp,nconnect=16 10.99.2.230:/lambdatest /mnt/test
$ cat /mnt/test/testing-from-lambda.txt

Hello from AWS Lambda via CNQ! - 2025-04-03T21:01:21

Debugging: If you experience any errors, you can view the output for debugging purposes:

aws logs filter-log-events --log-group-name /aws/lambda/cnq-nfs-lambda --limit 5

Why This Matters

Integrating AWS Lambda with CNQ enhances serverless capabilities, allowing for seamless stateful workflows without added complexity. CNQ’s multiprotocol support unlocks compatibility with legacy systems, delivering robust, scalable file storage—fast, simple, and serverless.

Happy coding!