Automatically migrating Images from Airtable to a CDN

October 27, 2024|pythonairtable

Are your Airtable images disappearing after just two hours? Airtable recently changed their attachment URL behavior, causing them to expire after 120 minutes. If you're using these URLs to display images on your website, you might be wondering what to do next. The solution? Migrate all your images to a CDN of your choice.

Not only will this fix the problem, but it can also speed up your site and reduce hosting costs. In this blog post, I'll show you how to automatically migrate your Airtable images to a CDN using Python and bunny.net as an example CDN.

Let's get started!

The Idea

To summarize, the steps to automatically migrate your Airtable images to a CDN are as follows:

Download all your images from Airtable to your local machine.
Optionally, optimize and resize the images to improve loading speed.
Upload the images to a CDN of your choice, such as bunny.net.
Update the URLs in your Airtable records to point to the newly uploaded images.

The Script

The following script will automatically download all the images from airtable, resize and optimize them, upload them to a CDN and update the airtable records with the new image URLs:

1from pyairtable import Table
2import requests
3import shutil
4from BunnyCDN.Storage import Storage
5import os
6import tinify
7
8# LOCAL CONFIG
9BASE_PATH = '<path-on-your-local-machine>'
10
11# AIRTABLE CONFIG
12API_KEY = '<your-airtable-api-key>'
13BASE_ID = '<your-airtable-base-id>'
14TABLE_NAME = '<your-table-name>'
15IMAGE_FIELD = '<your-image-field-name>'
16IMAGE_URL_FIELD = '<your-image-url-field-name>'
17
18# TINIFY CONFIG
19tinify.key = "<your-tinify-api-key>"
20
21# BUNNY CDN CONFIG
22BASE_BUNNY_URL = '<base-path-of-your-cdn>'
23BUNNY_STORAGE_CONFIG = {
24    "api_key": "<your-bunny-api-key>",
25    "zone_name": "<your-bunny-zone-name>",
26    "zone_region": "<your-bunny-zone-region>"
27}
28BUNNY_STORAGE_CLIENT = Storage(
29    BUNNY_STORAGE_CONFIG["api_key"],
30    BUNNY_STORAGE_CONFIG["zone_name"],
31    BUNNY_STORAGE_CONFIG["zone_region"]
32)
33
34def upload_file(client, filename, storage_path, local_upload_file_path=None):
35    return client.PutFile(
36        filename,
37        storage_path=storage_path,
38        local_upload_file_path=local_upload_file_path
39    )
40
41def download_image_from_airtable(record):
42  url = record["fields"][IMAGE_FIELD][0]["url"]
43  name_slug = record["fields"]["name"].replace("-", "").replace("  ", " ").replace(" ", "_").replace("!", "").lower()
44  img_type = record["fields"][IMAGE_FIELD][0]["type"].split("/")[1]
45
46  file_name = "%s_img.%s" % (name_slug, img_type)
47  path = '%s/%s' % (BASE_PATH, name_slug)
48  full_path = '%s/%s' % (path, file_name)
49
50  if not os.path.exists(path):
51    os.makedirs(path)
52
53  res = requests.get(url, stream = True)
54
55  if res.status_code == 200:
56      with open(full_path,'wb') as f:
57        shutil.copyfileobj(res.raw, f)
58
59      try:
60        source = tinify.from_file(full_path)
61        converted = source.convert(type=["image/webp"])
62        extension = converted.result().extension
63
64        file_name =  "%s_image.%s" % (name_slug, extension)
65        converted.to_file("%s/%s" % (path, file_name))
66      except Exception as e:
67        print(e)
68        pass
69
70      print('Image sucessfully Downloaded: ',file_name)
71  else:
72      print('Image Couldn\'t be retrieved')
73
74  return name_slug, file_name
75
76def upload_image_to_bunny_cdn( folder_name, file_name):
77  upload_file(
78    BUNNY_STORAGE_CLIENT,
79    file_name,
80    '%s/%s' % (folder_name, file_name),
81    '%s/%s' % (BASE_PATH, folder_name)
82  )
83
84  print("--- Image successfully uploaded ---")
85
86def update_airtable_image_url(record, name_slug, file_name):
87  table.update(record["id"], {IMAGE_URL_FIELD: BASE_BUNNY_URL + name_slug + "/" + file_name})
88
89if __name__ == "__main__":
90  table = Table(API_KEY, BASE_ID, TABLE_NAME)
91  all_records = table.all();
92
93  for record in all_records:
94    image_slug, image_file_name = download_image_from_airtable(record)
95    upload_image_to_bunny_cdn(image_slug, image_file_name)
96    update_airtable_image_url(record, thumbnail_slug, thumbnail_file_name)

In the following sections, I will explain each part of the script in detail.

Downloading the Images from Airtable

To automatically download all your images from airtable, you can use pyairtable - A python wrapper arround the airtable API. With this, you can easily configure your table with:

1table = Table(API_KEY, BASE_ID, TABLE_NAME)

To download an image, you'll need to provide the column name of the image field. Then you can get the url from it and download it manually:

1def download_image_from_airtable(record):
2  url = record["fields"][IMAGE_FIELD][0]["url"]
3  img_type = record["fields"][IMAGE_FIELD][0]["type"].split("/")[1]
4
5  res = requests.get(url, stream = True)
6

Automatically Optimizing the images

If you want to optimize the images before uploading them to the CDN, you can use the tinypng api. This allows you to simply call the from_file method to load a file and convert and optimize it with the convert method:

1source = tinify.from_file(full_path)
2converted = source.convert(type=["image/webp"])
3extension = converted.result().extension
4
5file_name =  "%s_image.%s" % (name_slug, extension)
6converted.to_file("%s/%s" % (path, file_name))

In this case I'm converting the image to webp to save even more space.

Uploading the Images to a CDN

This step is optional and will depend on the CDN you're using. In my case, I'm using bunny.net and bunny provides me with an API so that I don't have to upload the images manually. You can simply call:

1upload_file(
2  BUNNY_STORAGE_CLIENT,
3  file_name,
4  '%s/%s' % (folder_name, file_name),
5  '%s/%s' % (BASE_PATH, folder_name)
6)

Check out the documentation for more information on how to use the bunny API.

Updating the Airtable records with the new image URLs

The last step is to update the airtable records with the new image URLs. This is done by calling:

1def update_airtable_image_url(record, name_slug, file_name):
2  table.update(record["id"], {IMAGE_URL_FIELD: BASE_BUNNY_URL + name_slug + "/" + file_name})

Note that you will need to choose a column which already exists in you airtable for IMAGE_URL_FIELD, otherwise you will get an error.

Conclusion

In conclusion, migrating your Airtable images to a CDN can save you from the hassle of constantly updating attachment URLs and provide a faster and more reliable website for your visitors. With the use of Python and an example CDN like bunny.net, it's easier than ever to automate this process and ensure that your images are always accessible. By following the steps outlined in this blog post, you'll be able to migrate your images to a CDN with ease and enjoy the benefits of a faster and more secure website.