Serverless On-call duty notifier – Part 1

Development3 Comments

As many engineers in the industry, we have on-call duty. The on-call duty is defined at the beginning of each month and the list of the on-call engineers for each date can be found in an excel sheet. Well, this is nice but I want to get notified when I’m on-call 🙂

I’ve created a simple app that sends SMS for each one of the on-call engineers at 8am everyday so we won’t need to check the excel. For implementing the task I chose to use AWS. It has nice lambda functions that allows me to write and run python code without starting compute instances (EC2) and it has a nice SNS service that allows me to send notification in email, sms and etc.

So first of all let’s start with creating two tables in a database. The first one will hold the list of the engineers and their phone number and the second one will hole the list of the on-call engineers per day.

In order to fill the initial information to the tables, I wrote a simple script and executed from my computer:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
import boto3

dynamodb = boto3.resource("dynamodb")

if __name__ == "__main__":
    users = dynamodb.Table("oncall-notifier.names")
    with users.batch_writer() as batch:
        batch.put_item(Item={"Name": "Alexander", "Phone": "+972000000000"})
        batch.put_item(Item={"Name": "Danny", "Phone": "+972000000000"})
        batch.put_item(Item={"Name": "Moshe", "Phone": "+972000000000"})

    dates = dynamodb.Table("oncall-notifier.dates")
    with dates.batch_writer() as batch:
        batch.put_item(Item={"Date": "2017-06-28", "Names": ["Alexander", "Danny"]})
        batch.put_item(Item={"Date": "2017-06-29", "Names": ["Alexander", "Moshe"]})
        batch.put_item(Item={"Date": "2017-06-30", "Names" : ["Danny", "Moshe"]})

I turned off the auto-scaling feature and chose 2 units for read/write because we don’t really need performance for doing 2-3 queries per day.

Afterwards, I went to the IAM roles page and created a new role that allows readonly access for dynamodb, using SNS and executing lambda expressions:

support-duty-iam-role

Now, we need to create a lambda. The lambda function is written in Python and does the following:

  1. Get the names of the engineers that should be on call duty today.
  2. For each name, get their phone number.
  3. Send an SMS message to all retrieved phone numbers.

For accessing AWS services from python, we’ll use the boto3 library.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
from __future__ import print_function

import boto3
import datetime
import sys

def get_today_duty(dynamodb):
    today = str(datetime.datetime.now().date())
    print("Querying database for date: %s" % today)

    # Get todays people from dates table
    dates = dynamodb.Table("oncall-notifier.dates")

    response = dates.get_item(Key={ "Date" : today })
    if not("Item" in response.keys()):
        print("Failed with response:  %s" % response)
        return []

    item = response["Item"]["Names"]
    print("Found entry: %s" % item)
    return item

def get_phone_numbers(dynamodb, people):
    users = dynamodb.Table("oncall-notifier.names")

    phones = []
    for name in people:
        print("Querying user information for %s" % name)
        response = users.get_item(Key={ "Name" : name })
        if not ("Item" in response.keys()):
            print("Failed with response:  %s" % response)
            continue

        phones.append(response["Item"]["Phone"])

    return phones

def send_sms_message(people, phones):
    sns = boto3.client("sns")

    message = "On-call for today - %s" % ", ".join(people)
    for number in phones:
        print("Sending message to %s" % number)
        sns.publish(Message=message, PhoneNumber=number)

def lambda_handler(event, context):
    dynamodb = boto3.resource("dynamodb")
    people = get_today_duty(dynamodb)
    if len(people) == 0:
        sys.exit(1)

    phones = get_phone_numbers(dynamodb, people)
    if len(phones) == 0:
        sys.exit(1)

    send_sms_message(people, phones)

When creating the lambda, I chose 128MB memory (we don’t really use it) and the IAM role we’ve created in the previous step.

Now what left is creating a trigger for the lambda. For this purpose I’ll use CloudWatch scheduled event that will be configured to run the lambda each day at 9am local time (CloudWatch cron is UTC timezone so the actual value will be 6am).

cloud-watch-support-duty-event

And that’s it! our app is ready.

You can see a run example here (using CloudWatch logs):

cloud-watch-support-duty-run-example

– Alexander

Oh hi there 👋
It’s nice to meet you.

Sign up to receive awesome content in your inbox, as soon as it is published!