Getting around Heroku's 30 seconds timeout limit with Flask


Getting around Heroku's 30 seconds timeout limit with Flask

2016/08/24

Tags: flask heroku timeout H12 generators

Heroku has this 30s request timeout window in which if your app doesn’t send anything within 30 seconds, the request is cancelled and Application Error is returned.

Application Error in Heroku

With Flask, there’s an easy way though. Just use Flask generators to return data as soon as possible. Let’s consider an app which does some long calculations:

from flask import Flask, Response
import requests

app = Flask(__name__)

def some_long_calculation(number):
  '''
  here will be some long calculation using this number
  let's simulate that using sleep for now :)
  '''
  import time
  time.sleep(5)

  return number

@app.route('/')
def check():
    output = []
    for i in range(10):
      output.append(some_long_calculation(i))
    html = "<br/>".join(output)
    return Response(html, mimetype='text/html')

if __name__ == '__main__':
    app.run(host='0.0.0.0', port=8080, debug=True)

Now, this app will take around 50 seconds to return output. By this time, Heroku would have already shown us Application Error. Good news is that Flask supports incremental response. Let’s modify this code to use that:

from flask import Flask, Response
import requests

app = Flask(__name__)

def some_long_calculation(number):
  '''
  here will be some long calculation using this number
  let's simulate that using sleep for now :)
  '''
  import time
  time.sleep(5)

  return number

@app.route('/')
def check():
    def generate():
      for i in range(10):
        yield "<br/>"   # notice that we are yielding something as soon as possible
        yield str(some_long_calculation(i))
    return Response(generate(), mimetype='text/html')

if __name__ == '__main__':
    app.run(host='0.0.0.0', port=8080, debug=True)

With this, the initial response would come as soon as the first yield line is encountered and our app will not run into timeout errors.

But there’s another 55 seconds rolling window after initial response. As the doc states:

An application has an initial 30 second window to respond with a single byte back to the client. However, each byte transmitted thereafter (either received from the client or sent by your application) resets a rolling 55 second window. If no data is sent during the 55 second window, the connection will be terminated.

So, we have to make sure some_long_calculation() doesn’t take more than 55 seconds. To get past this, we need to use concurrency practices to check for progress periodically and return progress at least around 50 seconds to prevent timeout.

Skim through the following links for more in-depth information: