AWS Chatbot custom message – solution

Most DevOps people who set up AWS Chatbot integrations with other AWS services eventually start wondering how to send custom messages through Chatbot.

At this point I would to remind you that your life will be much easier if you give up on the idea and instead send your message directly to Slack using a web hook.

But if you want to see this to the end:

Tom Stroobants documented the general SNS message format that Chatbot expects and it looks like this:

{
  "version": "0",
  "time": "1970-01-01T00:00:00Z",
  "id": "00000000-0000-0000-0000-000000000000",
  "account": "[your real account id]",
  "region": "[a real region]",
  "source": "aws.[a service prefix e.g. ec2]",
  "detail-type": "[you can use this field for your message]",
  "resources": [],
  "detail": {}
}

As long as these fields are present in the message AWS Chatbot will forward the message to Slack, but will not display any more details other than the text in the “detail-type” field, and doubles up that text.

To make AWS Chatbot deliver a more detailed message, one has to format the message according to the AWS Events that Chatbot supports. Which means our messages will have to have a predefined “detail-type” and “source”.

To see examples of all message formats that Chatbot can display, to find one that we could co-opt for our purposes:

  1. Open the EventBridge console at https://console.aws.amazon.com/events/.
  2. In the navigation pane, choose Rules.
  3. Choose Create rule.
  4. Enter a name and description for the rule.
  5. For Define pattern, choose Rule with an event pattern.
  6. Hit Next.
  7. For Event source, leave it on AWS events
  8. Now you can browse all available events under Sample Event / AWS events.

You will quickly notice that the event names are quite specific, and you might not want to use “VoiceId Batch Fraudster Registration Action” for your custom message.

I found that the “AWS Health Event” is innocent enough to be reusable, and now I am able to send free form paragraphs using the following:

{
    "version": "0",
    "id": "00000000-0000-0000-0000-000000000000",
    "account": "[my AWS account number]",
    "time": "1970-01-01T00:00:00Z",
    "region": "us-east-1",
    "source": "aws.health",
    "detail-type": "AWS Health Event",
    "resources": [],
    "detail": {
      "eventDescription": [{
        "language": "en_US",
        "latestDescription": "Long form message\nMore lines"
      }]
    }
}

I hope somebody with good enough connections to the AWS Chatbot team will get more details out of them, right now their official line is “AWS Chatbot only supports AWS Services”. Help?

HTH, imre

j j j

AWS Force MFA example policy doesn’t work on Administrators – Fix

There are several example policies written by Amazon itself, and also by other security providers like Yubico that claim to enforce MFA use, but simply do not work on users who have AdministratorAccess policy.

Here is an actual example policy written by Amazon that actually works: https://docs.aws.amazon.com/IAM/latest/UserGuide/reference_policies_examples_aws_my-sec-creds-self-manage.html

j j j

LambdaAccessDenied error in AWS Load Balancer – Solution

Permission handling in ELB and Lambda is somewhat magical, some of the tools autoprovision permissions behind the scene, and some of them sometimes mess up.

I had a Lambda that I was invoking from a load balancer and it simply did not work. The only hint was “LambdaAccessDenied” in the ALB logs.

I had everything configured correctly. I have added a lambda permission for the entire elasticloadbalancing.amazonaws.com service to invoke my function. I had the proper target groups. I had even enabled AWS SAM to autoprovision the IAM roles. The Lambda function was firing correctly, I had logs to show that it was executing.

But I kept getting “502 Bad Gateway” from the load balancer and the logs kept showing LambdaAccessDenied.

I removed all the custom stuff I created. I removed the alias. I removed and re provisioned the entire lambda function. I removed and recreated the target group.

Eventually I removed the target group and the permission I created,
and provisioned an “Application Load Balancer” Trigger from the Lambda console. This created a new target group and a new resource-based policy under Permissions, and suddenly everything started working, even though the new entries looked exactly the same as the entries I created.

Since there are only five entries on Google that even mention this error message, I figured you might want to save some time and learn from my experience.

j j j