Serverless On-call duty notifier – Part 1

2017-06-28 Development 3 Comments 3 minute read

As many engineers in the industry, we have on-call duty. The on-call duty is defined at the beginning of each month and the list of the on-call engineers for each date can be found in an excel sheet. Well, this is nice but I want to get notified when I’m on-call 🙂

I’ve created a simple app that sends SMS for each one of the on-call engineers at 8am everyday so we won’t need to check the excel. For implementing the task I chose to use AWS. It has nice lambda functions that allows me to write and run python code without starting compute instances (EC2) and it has a nice SNS service that allows me to send notification in email, sms and etc.

So first of all let’s start with creating two tables in a database. The first one will hold the list of the engineers and their phone number and the second one will hole the list of the on-call engineers per day.

In order to fill the initial information to the tables, I wrote a simple script and executed from my computer:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
import boto3

dynamodb = boto3.resource("dynamodb")

if __name__ == "__main__":
    users = dynamodb.Table("oncall-notifier.names")
    with users.batch_writer() as batch:
        batch.put_item(Item={"Name": "Alexander", "Phone": "+972000000000"})
        batch.put_item(Item={"Name": "Danny", "Phone": "+972000000000"})
        batch.put_item(Item={"Name": "Moshe", "Phone": "+972000000000"})

    dates = dynamodb.Table("oncall-notifier.dates")
    with dates.batch_writer() as batch:
        batch.put_item(Item={"Date": "2017-06-28", "Names": ["Alexander", "Danny"]})
        batch.put_item(Item={"Date": "2017-06-29", "Names": ["Alexander", "Moshe"]})
        batch.put_item(Item={"Date": "2017-06-30", "Names" : ["Danny", "Moshe"]})

I turned off the auto-scaling feature and chose 2 units for read/write because we don’t really need performance for doing 2-3 queries per day.

Afterwards, I went to the IAM roles page and created a new role that allows readonly access for dynamodb, using SNS and executing lambda expressions:

support-duty-iam-role

Now, we need to create a lambda. The lambda function is written in Python and does the following:

  1. Get the names of the engineers that should be on call duty today.
  2. For each name, get their phone number.
  3. Send an SMS message to all retrieved phone numbers.

For accessing AWS services from python, we’ll use the boto3 library.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
from __future__ import print_function

import boto3
import datetime
import sys

def get_today_duty(dynamodb):
    today = str(datetime.datetime.now().date())
    print("Querying database for date: %s" % today)

    # Get todays people from dates table
    dates = dynamodb.Table("oncall-notifier.dates")

    response = dates.get_item(Key={ "Date" : today })
    if not("Item" in response.keys()):
        print("Failed with response:  %s" % response)
        return []

    item = response["Item"]["Names"]
    print("Found entry: %s" % item)
    return item

def get_phone_numbers(dynamodb, people):
    users = dynamodb.Table("oncall-notifier.names")

    phones = []
    for name in people:
        print("Querying user information for %s" % name)
        response = users.get_item(Key={ "Name" : name })
        if not ("Item" in response.keys()):
            print("Failed with response:  %s" % response)
            continue

        phones.append(response["Item"]["Phone"])

    return phones

def send_sms_message(people, phones):
    sns = boto3.client("sns")

    message = "On-call for today - %s" % ", ".join(people)
    for number in phones:
        print("Sending message to %s" % number)
        sns.publish(Message=message, PhoneNumber=number)

def lambda_handler(event, context):
    dynamodb = boto3.resource("dynamodb")
    people = get_today_duty(dynamodb)
    if len(people) == 0:
        sys.exit(1)

    phones = get_phone_numbers(dynamodb, people)
    if len(phones) == 0:
        sys.exit(1)

    send_sms_message(people, phones)

When creating the lambda, I chose 128MB memory (we don’t really use it) and the IAM role we’ve created in the previous step.

Now what left is creating a trigger for the lambda. For this purpose I’ll use CloudWatch scheduled event that will be configured to run the lambda each day at 9am local time (CloudWatch cron is UTC timezone so the actual value will be 6am).

cloud-watch-support-duty-event

And that’s it! our app is ready.

You can see a run example here (using CloudWatch logs):

cloud-watch-support-duty-run-example

– Alexander

Oh hi there 👋
It’s nice to meet you.

Sign up to receive a notification when new posts are published!

We don’t spam!

Getting the right Jenkins build number using Python

Serverless On-call duty notifier - Part 2
3 thoughts on “Serverless On-call duty notifier – Part 1”
  1. Pingback: Serverless On-call duty notifier – Part 2 – At a glance of a key
  2. Dylan

    Hey there! This is my 1st comment here so I just wanted to give a quick shout out and say I truly enjoy reading through your posts. Can you suggest any other blogs/websites/forums that deal with the same topics? Many thanks!

    Reply
  3. Youlanda Trio

    I am happy to be able to find good information in your articles.

    Reply
Leave a Reply