A tutorial on AWS SNS Message filtering using message attributes.
Amazon Simple Notification Service (Amazon SNS) is a fully managed messaging service for both application-to-application (A2A) and application-to-person (A2P) communication. SNS A2A functionality allows us to dispatch messages to a large number of subscribers for parallel processing using topics. The A2P functionality allows us to send messages to large audience via SMS, mobile push, and email.
In this article we will look into using go lang to publish messages to SNS topic and using message filtering to selectively trigger aws lambda.
Prerequisite:
- AWS cli and SAM cli installed and configured on the system
- Knowledge of aws cloudformation (YAML templates)
- Knowledge on basics of golang and deployment
How message filtering reduce infrastructure cost?
In the cloud-native applications it is common to use messaging brokers to pass message between services or to invoke the cloud functions (In AWS jargon lambda). Consider, we have a banking service that process the banking transactions. The banks customers need to be notified about every transactions. Here customers can chose to be notified via SMS on their cell phones or via email.
The accounting service generate the alert message push it to the SNS topic. If we don’t have filtering policy set up for the lambda invocation, for each of the messages published to the SNS, both lambda will be invoked. As AWS charges for lambda invocation time, the cost will be higher. But using message attribute we can create filter policy to selectively invoke the lambda function, which results in lower costs.
Creating security policy:
For creating message filtering policy is tricky. We need to create filterPolicy under properties section of topic subscription for each of the lambda, in SNS cloudformation file. The general format is as below:
SubscriptionName:
Type: 'AWS::SNS::Subscription'
Properties:
Endpoint: !ImportValue 'Fn::Sub': 'lambda-name'
FilterPolicy:
attributeName:
- attributeValue
Protocol: lambda
TopicArn: !Ref AlertSNSTopicIn the example above, attributeName is of type string and can be any text. Its type can be an object or array, and attributeValue can be of type supported by aws in message attributes.
Creating Resources in AWS Cloud Formation:
Lambda:
SAM template is used for deploying AWS Lambda. SAM templates are custom cloudformation templates to deploy ServerLess Applications written in go lang.
AWSTemplateFormatVersion: 2010-09-09
Transform: AWS::Serverless-2016-10-31
Resources:
LambdaEmailAlert:
Type: AWS::Serverless::Function
Properties:
CodeUri: build/
Handler: main
Role: !GetAtt LambdaExecutionRole.Arn
Runtime: go1.x
Timeout: 10
MemorySize: 128
EmailLambdaExecutionRole:
Type: AWS::IAM::Role
Properties:
FunctionName: !Sub 'test-lambda-email'
AssumeRolePolicyDocument:
Version: "2012-10-17"
Statement:
- Effect: "Allow"
Principal:
Service:
- "lambda.amazonaws.com"
Action:
- "sts:AssumeRole"
Path: "/"
ManagedPolicyArns:
- "arn:aws:iam::aws:policy/service-role/AWSLambdaBasicExecutionRole"Outputs:
TestSNSMessageAttributes:
Value: !GetAtt LambdaEmailAlert.Arn
Export:
Name: !Sub 'test-lambda-email'
Deploying Lambda:
go build -o build/main src/main.go
sam package \
--template-file sam.yml \
--s3-bucket <some-bucket-name> \
--output-template-file package.ymlSNS:
This cloudformation template creates the SNS Topic that invoke lambda on the basis of value of alert in the message attribute.
AWSTemplateFormatVersion: '2010-09-09'
Description: SNS topic for triggering notification lambdasResources:
AlertSNSTopic:
Type: AWS::SNS::Topic
Properties:
DisplayName: !Sub 'test-sns-message-sttributes'
SubscriptionEmail:
Type: AWS::SNS::Subscription
Properties:
Endpoint: !ImportValue
'Fn::Sub': 'test-lambda-email'
FilterPolicy:
alert:
Protocol: lambda
TopicArn: !Ref AlertSNSTopic
SubscriptionSMS:
Type: AWS::SNS::Subscription
Properties:
Endpoint: !ImportValue
'Fn::Sub': 'test-lambda-sms'
FilterPolicy:
alert:
- sms
Protocol: lambda
TopicArn: !Ref AlertSNSTopic
AlertSNSMessageAttributesPolicy:
Type: AWS::SNS::TopicPolicy
Properties:
Topics:
- !Ref AlertSNSTopic
PolicyDocument:
Version: '2012-10-17'
Statement:
- Effect: Allow
Principal:
AWS: '*'
Action:
- "SNS:Publish"
Resource: '*'
LambdaTriggerPermissionEmail:
Type: AWS::Lambda::Permission
Properties:
Action: 'lambda:InvokeFunction'
FunctionName: !ImportValue
'Fn::Sub': 'test-lambda-email'
Principal: "sns.amazonaws.com"
SourceArn: !Ref AlertSNSTopic
LambdaTriggerPermissionSMS:
Type: AWS::Lambda::Permission
Properties:
Action: 'lambda:InvokeFunction'
FunctionName: !ImportValue
'Fn::Sub': 'test-lambda-sms'
Principal: "sns.amazonaws.com"
SourceArn: !Ref AlertSNSTopic
Deploying SNS:
aws cloudformation \
create-stack \
--stack-name Test-sns-message-sttributes \
--template-body file://sns.ymlSending Messages to SNS Topic:
In go aws-sdk-go package is used to send messages to the SNS topics. The message attributes are the key value pairs. The message attribute, in aws-go-sdk expect the instance of map with string keys and value of type *sns.MessageAttributeValue. It expect the message attributes in following format.
messageAttributes:{
"alert": {
dataType:"String.Array",
stringValue:"[\\\"sms\\\"]
},
}The general code to create a message with associated message attributes is as below:
package mainimport (
"encoding/json"
"fmt"
"os""github.com/aws/aws-sdk-go/aws"
"github.com/aws/aws-sdk-go/aws/session"
"github.com/aws/aws-sdk-go/service/sns"
)func main() {
msg := "this message is for test email lambda"
topic := "arn:aws:sns:region:your-sns-topic-here" messageAttributes := make(map[string]*sns.MessageAttributeValue)
alert := []string{"email"} message, _ := json.Marshal(alert)
messageAttributes["alert"] = &sns.MessageAttributeValue{
DataType: aws.String("String.Array"),
StringValue: aws.String(string(message)),
}// Initialize a session that the SDK will use to load
// credentials from the shared credentials file
//(~/.aws/credentials).sess := session.Must(session.NewSessionWithOptions(session.Options{
SharedConfigState: session.SharedConfigEnable,
}))svc := sns.New(sess)result, err := svc.Publish(&sns.PublishInput{
Message: &msg,
TopicArn: &topic,
MessageAttributes: messageAttributes,
})if err != nil {
fmt.Println(err.Error())
os.Exit(1)
}fmt.Println(*result.MessageId)
}
If we run this code with go run main.go the first lambda will be invoked. Now if we replace “alert := []string{“email”}” with “alert := []string{“sms”}” in the code and run again, then second lambda get executed.
References:
