Is concurrent computing important for web development?

Question

Let's say I have a web application running on S servers with an average of C cores each. My application is processing an average of R requests at any instant. Assuming R is around 10 times larger than S * C, won't benefits from spreading the work of a request across multiple cores be minimal since each core is processing around 10 requests already?

If I'm correct why does this guy say concurrency is so important to the future of Python as a language for web development?

I can see many reasons why my argument would be incorrect. Perhaps the application receives a few very-difficult-to-process requests that are outnumbered by available cores. Or perhaps there is a large variation in the difficulty of requests, so it's possible for one core to be unlucky and be given 10 consecutive difficult requests, with the result being that some of them take much longer than is reasonable. Given that the guy who wrote the above essay is so much more experienced than I am, I think there's a significant chance I'm wrong about this, but I'd like to know why.

Answer 1

In the hypothetical circumstances you design, with about 10 requests "in play" per core, as long as the request-to-core assignment is handled sensibly (probably even the simplest round-robin load balancing will do), it's just fine if each request lives throughout its lifetime on a single core.

Point is, that scenario's just ONE possibility -- heavy requests that could really benefit (in terms of lower latency) from marshaling multiple cores per request are surely an alternative possibility. I suspect that on today's web your scenario is more prevalent, but it sure would be nice to handle both kinds, AND "batch-like" background processing ones too.... especially since the number of cores (as opposed to each core's speed) is what's increasing, and what's going to keep increasing, these days.

Far be it from me to argue against Jacob Kaplan-Moss's wisdom, but I'm used to getting pretty good concurrency, at my employer, in nicer and more explicit AND trasparent ways than he seems to advocate -- mapreduce for batch-like jobs, distributed-hashing based sharding for enrolling N backends to shard the work for 1 query, and the like.

Maybe I just don't have enough real-life experience with (say) Erlang, Scala, or Haskell's relatively-new software transactional memory, to see how wonderfully they scale to high utilization of tens or hundrends of thousands of cores on low-QPS, high-work-per-Q workloads... but it seems to me that the silver bullet for this scenario (net of the relatively limited subset of cases where you can turn to mapreduce, pregel, sharding, etc) has not yet been invented in ANY language. With explicit, carefully crafted architecture Python is surely no worse than Java, C# or C++ at handling such scenarios, in my working experience at least.