Why is Akamai CDN expensive

Case Study: Zillow Case

The challenge

Zillow processes more than 3 million new images daily, including ad photos, lender and broker profile photos, and home project images on the Zillow Digs website. "In times of high utilization, we receive 17,000 image requests per second from desktop and mobile clients," says Nick Michal, Unix Systems Engineering Manager at the Zillow Group.

As Zillow grew in popularity and brokers began posting high resolution images in their advertisements, the company's outdated imaging system could no longer meet the need. The system was in hosted data centers. Images were downloaded from a single queue, stored on a network storage device (NAS) in pyramidal TIFF format, and delivered to a content delivery network (CDN) through a local Squid service. "That was expensive and we had to rely on a high cache hit rate from the CDN. Otherwise we couldn't process images effectively. We were at the limit of our capacity every day," explains Michal.

Zillow also suffered from image processing performance issues as some images were uploaded manually while others came from bulk feeds that needed to be downloaded. The speed at which new images came out of the bulk feeds was often unpredictable, and some image sources allowed much faster download speeds with more simultaneous downloads than other sources. A slow or problematic image at the beginning of the queue could prevent other images from being downloaded. "At Zillow, we can't afford bandwidth problems because when users visit the website and a picture is missing, they don't even look at the corresponding ad," said Feroze Daud, senior software development engineer at the Zillow Group. "It's very frustrating for users."

Also, the tool that Zillow used to process the images for saving in the pyramid-like TIFF format was outdated and not easily expandable. "Improvements in image quality, for example by removing fixed color borders, were not easy to implement," explains Daud. Disaster recovery was another big problem. "There was a risk because everything was hosted in a single data center," says Michal.

Reasons for Amazon Web Services

To meet the challenges of scalability, performance and disaster recovery of the imaging system, Zillow decided to move to a cloud-based infrastructure. "Considering the total cost and the ease of administration, the cloud actually makes sense," explains Michal. After evaluating various cloud technologies, Zillow decided on Amazon Web Services (AWS). "AWS has been offering cloud solutions for a long time and is a leader in this area," said Michal. "Also, some of the companies we bought were already using AWS."

The company migrated image hosting and distribution from a physical facility to AWS using Amazon Elastic Compute Cloud Instances (Amazon EC2) and Amazon Simple Storage Service (Amazon S3) for storing image assets. Zillow currently stores nearly 100 TB of data in Amazon S3, including 300 million images and more than 1 billion objects. "In a conventional file system it is hardly possible to count billions of objects," explains Michal. "We'd have to split these objects across multiple file systems, which would be an administrative nightmare. The scalability of Amazon S3 seemed like the right technology to us."

Zillow also began using AWS Elastic Beanstalk, a service for deploying and scaling web applications and services. The developers upload their code to Elastic Beanstalk, which then automatically takes care of the provisioning - from capacity provisioning, load balancing and auto scaling to monitoring the status of the application. The company uses an Elastic Beanstalk worker environment to run a Python imaging library of custom code. "Since we're ingesting data in a random manner, running feeds that dump a lot of work into the system at once, we need to upscale our image conversion suite," says Daud. "AWS Elastic Beanstalk is the easiest way to do this - as opposed to a series of static instances or trying to write our own configuration for auto scaling."

After that, the organizations moved most of the CDN processing load from Akamai to Amazon CloudFront, a content delivery web service that further distributes the Zillow website content to users. "AWS CloudFront is significantly cheaper than Akamai and can be easily integrated into Amazon S3," explains Michal. Zillow also uses Amazon CloudWatch to monitor some of its cloud resources.

Zillow uses a download server (DLS) in the data center to manage image download requests from ad feeds and an Amazon Elastic Beanstalk REST API as a front-end service in the cloud for the DLS. This service moves all download requests to a feed-based Amazon Simple Queue Service (Amazon SQS) message queue service. "With SQS we have a queuing system without having to support the infrastructure itself," says Michal.

A throttled downloader controls the speed and simultaneity with which Zillow downloads images for each feed source so that the company can use image providers that support fast downloads while not overwhelming providers that do not support them. If the image download is successful, Zillow writes the original image for image processing in Amazon S3.

For image processing, Zillow uses the original images stored in S3 and processes them with various image quality methods, while a set of standard sizes is generated for each image. All images are served outside of Amazon S3 and cached in Amazon CloudFront. The company provides an average of 15,000 images per second.

The advantages

Using AWS, Zillow can provide a better user experience for potential buyers and tenants, real estate agents, and other website visitors. "By moving to AWS, we no longer have to worry about cache overloads or capacity issues." We have the scalability and performance we need to deliver high quality real estate imagery, which is critical to the Zillow user experience. " "Daud explains. Zillow can scale the downloading and processing of images to handle varying amounts of images coming in every day. And since image downloads from each feed source are now independent, Zillow can use sources that support high bandwidth and parallel downloads. while at the same time throttling sources that do not support it, and Amazon S3 gives the company almost unlimited object storage, eliminating the need to order and install servers or hard drives to increase capacity.

With the help of Amazon CloudFront and Amazon S3, Zillow can rely on the performance of its imaging system. "We have a lot more bandwidth than before, so we don't even think about it," says Michal. "And we definitely don't worry about S3 running out of capacity."

Zillow also reduced operational costs by migrating their vision and delivery system to AWS. "With Amazon CloudFront, we pay less than half of what we previously paid for our CDN," emphasizes Michal. "We no longer have to spend money on major NAS device upgrades."

The company has used Amazon S3 and Amazon CloudFront to improve the availability of its imaging system. "With S3, objects are replicated three times in a region. So if an Availability Zone really should fail, the data traffic could still be made available to the users without any development effort on our part," explains Michal.

Disaster recovery has also been improved. "We definitely benefit from the geographic distribution of AWS," says Michal. "There are multiple AWS Regions and Availability Zones in these regions, so not only can we generate dynamic content that is close to the users, but we can also improve our disaster recovery capabilities."

Zillow is now more flexible in responding to scalability needs. "I can always set up Amazon EC2 instances to make a major application version change, or I can create a new Amazon CloudFront distribution with just a few clicks," explains Daud. "In the end, AWS accelerated our operations."

The organization also has better insights into system performance. "There were some delays in the image processing, and the early versions of our cloud apps didn't provide enough metrics that we couldn't see which component of the pipeline was causing the delay," says Michal. "After we started monitoring latency using Amazon CloudWatch, the causes were much clearer to us and we could take appropriate action to fix it."

Zillow will continue to look for additional ways to move services to the cloud. "When we migrated to AWS, CloudFront was relatively new and we thought we were taking a risk," said Michal. "But it has proven itself. In the future, we will consider AWS for new projects and services. Our experience with AWS has convinced us."

About Zillow

Every day tens of millions of buyers, sellers, renters and landlords, as well as brokers and property managers, use the Zillow website to browse home listings, purchase mortgages, and find information on 110 million homes across the United States.

Benefits of AWS

  • More than 50% monthly savings
  • Improved availability and scalability
  • Better insight into system performance

AWS services used

Amazon EC2

The Amazon Elastic Compute Cloud (Amazon EC2) web service provides secure, scalable computing capacities in the cloud. The service is designed to make cloud computing easier for developers.

Additional Information "

Amazon CloudWatch

Amazon CloudWatch is a transparent monitoring and observation service for DevOps technicians, developers, Site Reliability Engineers (SRE) and IT managers.

Additional Information "

Amazon S3

Amazon Simple Storage Service (Amazon S3) is an object storage service that offers industry-leading scalability, data availability, security, and performance.

Additional Information "

Amazon SQS

Amazon Simple Queue Service (SQS) is a fully managed message queue service that enables the decoupling and scaling of microservices, distributed systems, and serverless applications.

Additional Information "

Elastic Beanstalk

AWS Elastic Beanstalk is an easy-to-use service for deploying and scaling web applications and services developed using Java, .NET, PHP, Node.js, Python, Ruby, Go and Docker on familiar servers such as Apache, Nginx, Passenger and IIS .

Additional Information "

Amazon CloudFront

Amazon CloudFront is a fast content delivery service (CDN) that delivers data, videos, applications and APIs securely, with low latency, high transfer speeds and within a developer-friendly environment to customers around the world.

Additional Information "

First steps

Companies of all sizes and in all industries use AWS to be more successful. Contact our experts and embark on your own AWS cloud journey today.