According to Jeff Dean, Google Senior Fellow, “back-of-the-envelope calculations are estimates you create using a combination of thought experiments and common performance numbers to get a good feel for which designs will meet your requirements”

• Latency numbers every programmer should know

  

  We get the following conclusions:

  • Memory is fast but the disk is slow.
  • Avoid disk seeks if possible.
  • Simple compression algorithms are fast.
  • Compress data before sending it over the internet if possible.
  • Data centers are usually in different regions, and it takes time to send data between them.

• Availability numbers

  A service level agreement (SLA) is a commonly used term for service providers.

  

• Example: Estimate Twitter QPS and storage requirements

  Assumptions:

  • 300 million monthly active users.
  • 50% of users use Twitter daily.
  • Users post 2 tweets per day on average.
  • 10% of tweets contain media.
  • Data is stored for 5 years.

  Estimations:

  • Query per second (QPS) estimate:

    Daily active users (DAU) = 300 million * 50% = 150 million

    Tweets QPS = 150 million * 2 tweets / 24 hour / 3600 seconds = ~3500

    Peak QPS = 2 * QPS = ~7000

  • We will only estimate media storage here.

    Average tweet size:

    • tweet_id 64 bytes
    • text 140 bytes
    • media 1 MB

    Media storage:

    150 million * 2 * 10% * 1 MB = 30 TB per day

    5-year media storage: 30 TB * 365 * 5 = ~55 PB