Building AI Agent Tools using OpenAI and Python
Integrating Calendly, Google Calendar, Google Meet and Sendgrid to create an AI Agent for Scheduling
Over the past few decades there’s been a SaaS platform created for almost every task and workflow you could possibly imagine.
Need a ride?
Online Payments?
Launch a pop-up store?
Get a D-List celebrity to wish your Grandma Happy Birthday?
We’ve got the platform for you.
But the problem with the SaaS-ification of everything is that it comes with a couple of steep and innate drawbacks:
- It’s Utilitiarian. The greatest software for the greatest number of people (or, more accurately, for those with the greatest willingness to pay)
- It’s Fractured. Every app wants to do everything, but differentiation and competition means you’ll always get some functionality you need in App A, some in App B, and yet more in App C.
And it’s not a small problem either — a Harvard study estimates that you lose 9% of your day purely to switching tabs.
And I absolutely believe it too, because it happens all the time.
I get an email for someone who wants to schedule a meeting, then need to go grab my Calendly link, then open back up into Gmail to send it, update the meeting with notes in Calendar, change the time, etc.
But this is exactly where the best part of AI Agents comes in — now I can start to build my own custom toolkits that are based on how I actually work, not how most people like me tend to work. And because the LLM can figure out to interact with these functions for me, I can do it faster than ever.
Where before I would have to spend time build out the logic, then the front-end, then iterate over and over to get the flow and manage the edge cases. Now I can just focus on building out the logic.
So let’s do that together.
If you want to skip to the code, you can get it right here. If you’re here for the journey, read on!
What We’ll Cover
In this tutorial, we’re going to be building a toolkit for you to use in managing a personal scheduler.
Once you’re done, you’ll be able to use this in your own AI Agent, as part of an AI pipeline, or maybe as part of system that helps in managing touch-points with new customers based on their feedback.
For myself, I adapted the code to make a GPT that I can talk to directly to handle setting up meetings rather than having to navigate through 4 separate apps.
To do this, we’ll be integrating APIs from Calendly, Sendgrid, Google Meet and Google Calendar, and using Vaul to automatically turn these functions into OpenAI friendly tools.
To do this, we’ll be creating a set of functions using the Calendly, Sendgrid, Google Meet and Google Calendar APIs, and automatically generating the JSON schema using Vaul to turn these functions into tools.
Tools and Tech
- Calendly API
- Sendgrid API
- Google Meet API
- Google Calendar API
- Vaul
First Steps
There’s a few permissions, tokens and configurations you will have to adjust to get each of the APIs to work properly. These are the guides that I used to get things set up, but if you find something missing, please leave a comment below to help others figure it out as well!
If you’re also using a service account for interacting with Google Resources, place your JSON file in the main project directory
Setting the environment
To make sure that you have everything you’ll need, create your .env file in that main directory of your new project as well. You’ll get most of these from the links/guides in the services above.
OPENAI_API_KEY=`openai-api-key`
GOOGLE_APPLICATION_CREDENTIALS=`your-credential-file.json`
GOOGLE_CALENDAR_DELEGATED_USER=`the-email-of-the-google-account-you-are-using`
CALENDLY_PERSONAL_ACCESS_TOKEN=`personal-access-token`
CALENDLY_ORGANIZATION_ID=`https://api.calendly.com/organizations/your-organization-id`
SENDGRID_API_KEY=`sendgrid-api-key`
MAIL_DEFAULT_SENDER=`email-account-verified-in-sendgrid`Building the functions
Let’s get started with installing our libraries:
pip install requests==2.31.0 google-apps-meet google-auth-httplib2 google-auth-oauthlib sendgrid google-api-python-client google-authOnce those are finished, you can go ahead and create your main.py file (or whatever name you prefer).
To get into the actual code a little bit quicker, we’ll get the overhead of loading our environment variables, scopes the the Google API and creating credentials out of the way. Keep in mind that as I am doing this as a personal project on my personal Google resources, I will be using a service account file and delegating authority to access those resources using the with_subject() method for Google credentials.
import os
from datetime import datetime
from dotenv import load_dotenv
from vaul import tool_call
from googleapiclient.discovery import build
from google.apps import meet_v2
from google.oauth2 import service_account
load_dotenv('.env')
SERVICE_ACCOUNT_FILE = f'./{os.environ.get("GOOGLE_APPLICATION_CREDENTIALS")}'
SCOPES = [
'https://www.googleapis.com/auth/calendar',
'https://www.googleapis.com/auth/calendar.events',
'https://www.googleapis.com/auth/admin.directory.resource.calendar',
'https://www.googleapis.com/auth/meetings.space.created'
]
credentials = service_account.Credentials.from_service_account_file(SERVICE_ACCOUNT_FILE, scopes=SCOPES)
credentials = credentials.with_subject(os.environ.get('GOOGLE_CALENDAR_DELEGATED_USER'))With that done, we can dive right into creating our Google Calendar functions. I’m using the Vaul @tool_call decorator that will allow these to be used with the OpenAI session later, and also added in a get_current_datetime function, as the ChatGPT won’t natively have access to it. While you could also add this to your system message, for the sake of simplicity I’ve added it as it’s own separate tool.
# ...Previous Code
@tool_call
def get_current_datetime() -> dict:
"""Get the current datetime."""
return {
'datetime': datetime.utcnow()
}
@tool_call
def list_events(time_min: str, time_max: str) -> dict:
"""List all events in a calendar
:param time_min: The minimum time to list events as a datetime string (ie. '2022-01-01T00:00:00Z')
:param time_max: The maximum time to list events as a datetime string (ie. '2022-01-01T00:00:00Z')
"""
service = build('calendar', 'v3', credentials=credentials)
events_result = service.events().list(
calendarId='primary',
timeMin=time_min,
timeMax=time_max,
singleEvents=True,
orderBy='startTime'
).execute()
events = events_result.get('items', [])
if not events:
return {'message': 'No events found.'}
return {'events': events}
@tool_call
def create_event(summary: str, description: str, start_time: str, end_time: str) -> dict:
"""Create a new event in a calendar.
:param summary: The event summary
:param description: The event description
:param start_time: The start time of the event as a datetime string (ie. '2022-01-01T00:00:00Z')
:param end_time: The end time of the event as a datetime string (ie. '2022-01-01T00:00:00Z')
"""
service = build('calendar', 'v3', credentials=credentials)
event = {
'summary': summary,
'description': description,
'start': {
'dateTime': start_time,
'timeZone': 'UTC',
},
'end': {
'dateTime': end_time,
'timeZone': 'UTC',
},
}
event = service.events().insert(calendarId='primary', body=event).execute()
return event
@tool_call
def get_event(event_id: str) -> dict:
"""Get an event by ID.
:param event_id: The ID of the event
"""
service = build('calendar', 'v3', credentials=credentials)
event = service.events().get(calendarId='primary', eventId=event_id).execute()
return event
@tool_call
def update_event(event_id: str, summary: str, description: str, start_time: str, end_time: str) -> dict:
"""Update an event.
:param event_id: The ID of the event
:param summary: The event summary
:param description: The event description
:param start_time: The start time of the event as a datetime string (ie. '2022-01-01T00:00:00Z')
:param end_time: The end time of the event as a datetime string (ie. '2022-01-01T00:00:00Z')
"""
service = build('calendar', 'v3', credentials=credentials)
event = {
'summary': summary,
'description': description,
'start': {
'dateTime': start_time,
'timeZone': 'UTC',
},
'end': {
'dateTime': end_time,
'timeZone': 'UTC',
},
}
event = service.events().update(calendarId='primary', eventId=event_id, body=event).execute()
return event
@tool_call
def delete_event(event_id: str) -> dict:
"""Delete an event.
:param event_id: The ID of the event
"""
service = build('calendar', 'v3', credentials=credentials)
service.events().delete(calendarId='primary', eventId=event_id).execute()
return {
'message': 'Event deleted.'
}With our Google Calendar integration now handled, the next step is to add Calendly. Calendly provides links to invites for different types of events, and I’d prefer to not have to constantly come back and update the code whenever I create/update/delete an event.
Because of that, I’ll create a function that retrieves all the event types for my organization:
@tool_call
def get_calendly_links() -> dict:
"""Get all Calendly links."""
headers = {
'Authorization': f"Bearer {os.environ.get('CALENDLY_PERSONAL_ACCESS_TOKEN')}",
'Content-Type': 'application/json'
}
params = {
'organization': os.environ.get('CALENDLY_ORGANIZATION_ID')
}
response = requests.get(
"https://api.calendly.com/event_types",
headers=headers,
params=params
)
if response.status_code > 299:
return {
'message': "An error occurred while fetching Calendly links."
}
return response.json()With access to the links, now we’ll set up a preset email function to be able to send them out. I’ll use Sendgrid purely from a familiarity standpoint, but feel free to swap out for your preferred provider.
@tool_call
def send_invite_email(to_email: str, calendly_link: str) -> dict:
"""Send a personalized email using SendGrid.
:param to_email: The email address to send the invite to
:param calendly_link: The Calendly link to include in the email
"""
sg = sendgrid.SendGridAPIClient(api_key=os.environ.get('SENDGRID_API_KEY'))
from_email = Email(email=os.environ.get('MAIL_DEFAULT_SENDER'), name=os.environ.get('MAIL_DEFAULT_SENDER_NAME'))
subject = "Let's Schedule a Meeting 📅"
# Set up the HTML email content with a styled button
content = f"""
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="UTF-8">
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<link rel="preconnect" href="https://fonts.googleapis.com">
<link rel="preconnect" href="https://fonts.gstatic.com" crossorigin>
<link href="https://fonts.googleapis.com/css2?family=Inter:wght@400;700&display=swap" rel="stylesheet">
<style>
html {{
background-color: #f9fafb;
}}
body {{
font-family: 'Inter', sans-serif;
color: #000000;
}}
.container {{
max-width: 600px;
margin: 0 auto;
padding: 20px;
}}
h1, h2, h3 {{
color: #15803d;
font-family: 'Inter', sans-serif;
}}
p, ul {{
line-height: 1.5;
}}
a {{
color: #fff;
text-decoration: none;
font-weight: 600;
margin: 24px 0;
}}
.btn {{
display: inline-block;
background-color: #15803d;
color: #fff;
padding: 10px 20px;
border-radius: 5px;
text-align: center;
text-decoration: none;
}}
.ii a[href] {{
color: #fff !important;
text-decoration: none;
font-weight: 600;
margin: 24px 0;
}}
p {{ margin: 0; padding: 0; }}
</style>
</head>
<body>
<div class="container">
<div style="margin-top: 50px;">
<h2>Let's Schedule a Meeting</h2>
<p>Let's set up a time to chat. You can schedule a meeting with me using the button below:</p>
<p><a href="{calendly_link}" class="btn">Schedule a Meeting</a></p>
<p>If you did not request this email, please ignore it.</p>
<p>Best regards,<br>
{from_email.name}</p>
<br/>
<p style="font-size: 12px">
If you’re having trouble clicking the "Schedule a Meeting" button, copy and paste the URL below into your web browser:
<br>
<a href="{calendly_link}">{calendly_link}</a>
</p>
</div>
</div>
</body>
</html>
"""
message = Mail(
from_email=from_email,
to_emails=To(to_email),
subject=subject,
html_content=Content("text/html", content.strip())
)
sg.send(message)
return {
'message': 'Email sent.'
}With the emails now taken care of, there’s one last thing I’d like to have — creating a Google Meet link on the fly. As I grab impromptu rooms constantly, having this at the ready will be a huge help.
@tool_call
def get_google_meet_link() -> dict:
"""Get a Google Meet link."""
client = meet_v2.SpacesServiceClient(credentials=credentials)
request = meet_v2.CreateSpaceRequest()
response = client.create_space(request)
return {
'message': 'Google Meet link created.',
'meeting_uri': response.meeting_uri
}
Making the Toolkit
Our functions are now finished, and we can start to create our toolkit. A toolkit is a basic way for us to view and manipulate the available tools for our agent. I’ve set it up as a dictionary:
# ... Previous code
toolkit = {
'list_events': list_events,
'create_event': create_event,
'get_event': get_event,
'update_event': update_event,
'delete_event': delete_event,
'get_calendly_links': get_calendly_links,
'send_invite_email': send_invite_email,
'get_google_meet_link': get_google_meet_link,
'get_current_datetime': get_current_datetime,
}Now we’ll setup the OpenAI session, and plug the toolkit directly in before creating the code to manage the run.
# ...Other imports
from openai import OpenAI
openai_session = OpenAI(
api_key=os.environ.get("OPENAI_API_KEY")
)
# ...Previous Code...
# Send a message to the OpenAI API
response = openai_session.chat.completions.create(
model="gpt-3.5-turbo",
messages=[
{
"role": "system",
"content": "You are a Google Calendar bot that can list a users calendars, list events, create events, get events, update events, and delete events. You can also get Calendly links and send an email with a Calendly link. Finally, you can get a Google Meet link."
},
{
"role": "user",
"content": "List all of my Calendly links"
}
],
tools=[{
"type": "function",
"function": value.tool_call_schema
} for key, value in toolkit.items()],
)
# Identify the tool call, if any
try:
tool_call = response.choices[0].message.tool_calls[0].function.name
except AttributeError:
tool_call = None
# Run the tool if it exists
if tool_call and tool_call in toolkit:
tool_run = toolkit[tool_call].from_response(response, throw_error=False)
print(tool_run)Conclusion
And you’re done!
In a single file, and just under 350 lines of code, you now have a robust toolkit for managing your schedule.
Hopefully you found this tutorial helpful — and if you did, please be sure to add some applause and share.
If you have any other tools you’d like to see or questions about how to work with them, and I’d be happy to try and help.
Happy Hacking!
