Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Why is flask url_for too slow

Tags:

python

flask

I tested it with this code:

from gevent import wsgi, monkey; monkey.patch_all()
from flask import Flask, url_for

app = Flask(__name__)

@app.route('/<int:n>')
def index(n):
    for i in xrange(n):
        url = url_for('index', n=i)
    return url 

wsgi.WSGIServer(('', 8000), app).serve_forever()

Results:

  • /1 Requests per second: 2721.94 [#/sec] (mean)
  • /10 Requests per second: 1080.16 [#/sec] (mean)
  • /100 Requests per second: 144.66 [#/sec] (mean)
like image 998
imbolc Avatar asked May 23 '13 12:05

imbolc


People also ask

Why is my flask app running slow?

When Flask app runs slow we need to identify what is the bottleneck. It can be an overloaded database, unresponsive external API, or heavy, CPU-intensive computation. This is the whole recipe on how to speed up Flask - find the source of sluggish performance. After the bottleneck is identified you can fight an underlying cause.

What is URL_for in flask?

Definition of Flask url_for. Flask url_for is defined as a function that enables developers to build and generate URLs on a Flask application. As a best practice, it is the url_for function that is required to be used, as hard coding the URL in templates and view function of the Flask application tends to utilize more time during modification.

Why is my server running so slow?

Profiling the run on the server might turn up an unexpected hot spot, or more likely, unexpected I/O delays. If there's something using tremendous amounts of memory, which could cause your program to slow waiting for paging or swapping. It might also be triggering heavy garbage collection activity.

How to use variable rules in flask for dynamic routing?

However, this can be solved using the flask with something called dynamic routing . We shall now look at a better approach using Variable Rules. We will add a <variable name> with each route. Optionally, we can also define the converter with each variable name <converter: variable name>. By default, the converter is String.


1 Answers

Indeed it is kinda slow.

Good news is time complexity is linear, O(1).

Below is cProfile dump

If I were a flask developer, I'd look at why url_for calls both urljoin and urlsplit. If I understand the werkzeug code correctly, it performs validation on the resulting url.

         13726316 function calls (13526316 primitive calls) in 16.918 seconds

   Ordered by: standard name

   ncalls  tottime  percall  cumtime  percall filename:lineno(function)
   400000    0.272    0.000    0.453    0.000 <string>:8(__new__)
   100000    0.120    0.000    0.140    0.000 app.py:1484(inject_url_defaults)
   200000    0.762    0.000    1.052    0.000 datastructures.py:316(__init__)
   100000    0.132    0.000    0.395    0.000 globals.py:16(_lookup_object)
   100000    0.913    0.000   16.996    0.000 helpers.py:250(url_for)
   300000    0.527    0.000    0.846    0.000 local.py:156(top)
   100000    0.136    0.000    0.645    0.000 local.py:289(_get_current_object)
   100000    0.112    0.000    0.901    0.000 local.py:333(__getattr__)
   300000    0.264    0.000    0.320    0.000 local.py:66(__getattr__)
   200000    0.059    0.000    0.059    0.000 routing.py:1199(update)
   200000    0.147    0.000    0.147    0.000 routing.py:1455(get_host)
400000/200000    0.802    0.000    6.297    0.000 routing.py:1520(_partial_build)
   200000    1.791    0.000   14.382    0.000 routing.py:1541(build)
   400000    0.153    0.000    0.153    0.000 routing.py:1601(<genexpr>)
   200000    1.830    0.000    5.181    0.000 routing.py:701(build)
   200000    0.275    0.000    0.275    0.000 routing.py:743(suitable_for)
   200000    0.256    0.000    0.256    0.000 routing.py:928(to_url)
   400000    0.935    0.000    2.816    0.000 urlparse.py:128(urlparse)
   400000    1.010    0.000    1.428    0.000 urlparse.py:159(urlsplit)
   200000    0.175    0.000    0.274    0.000 urlparse.py:214(urlunparse)
   200000    0.099    0.000    0.099    0.000 urlparse.py:224(urlunsplit)
   200000    1.961    0.000    5.637    0.000 urlparse.py:242(urljoin)
     5263    0.004    0.000    0.019    0.000 urlparse.py:62(clear_cache)
   400000    0.483    0.000    0.783    0.000 urls.py:36(_quote)
   400000    0.509    0.000    1.534    0.000 urls.py:369(url_quote)
   100000    0.068    0.000    0.068    0.000 wrappers.py:85(blueprint)
   505263    0.235    0.000    0.235    0.000 {built-in method __new__ of type object at 0x10d044248}
   200000    0.101    0.000    0.169    0.000 {getattr}
   100000    0.114    0.000    0.114    0.000 {hasattr}
  2000000    0.642    0.000    0.642    0.000 {isinstance}
   505263    0.073    0.000    0.073    0.000 {len}
   200000    0.041    0.000    0.041    0.000 {method 'add' of 'set' objects}
   600000    0.127    0.000    0.127    0.000 {method 'append' of 'list' objects}
     5263    0.015    0.000    0.015    0.000 {method 'clear' of 'dict' objects}
        1    0.000    0.000    0.000    0.000 {method 'disable' of '_lsprof.Profiler' objects}
     5263    0.003    0.000    0.003    0.000 {method 'find' of 'str' objects}
   100000    0.072    0.000    0.072    0.000 {method 'find' of 'unicode' objects}
   700000    0.231    0.000    0.231    0.000 {method 'get' of 'dict' objects}
   400000    0.105    0.000    0.105    0.000 {method 'iteritems' of 'dict' objects}
   200000    0.116    0.000    0.116    0.000 {method 'join' of 'str' objects}
   200000    0.157    0.000    0.157    0.000 {method 'join' of 'unicode' objects}
   200000    0.134    0.000    0.134    0.000 {method 'lstrip' of 'unicode' objects}
   300000    0.052    0.000    0.052    0.000 {method 'pop' of 'dict' objects}
   200000    0.079    0.000    0.079    0.000 {method 'remove' of 'list' objects}
   400000    0.249    0.000    0.249    0.000 {method 'rstrip' of 'str' objects}
   200000    0.183    0.000    0.183    0.000 {method 'split' of 'str' objects}
   400000    0.339    0.000    0.339    0.000 {method 'split' of 'unicode' objects}
   300000    0.056    0.000    0.056    0.000 {thread.get_ident}

I have 2-3 practical solutions for you:

  • cache (most common) url_for results
  • offload url_for computation to client/js
  • hardcode url generation

If you have RESTful API with numeric id's the last option may look like this:

datum_url_template = url_for("datum", n=999)
def url_for_datum(n):
    return datum_url_template.replace("999", str(n))

foobar_url_template = url_for("foobar", n=777)
def url_for_foobar(n):
    return foobar_url_template.replace("777", str(n))
like image 165
Dima Tisnek Avatar answered Sep 23 '22 21:09

Dima Tisnek