Embedded web server with integrated XML parser

Question

I have a requirement to integrate a web server into an embedded device running Linux and am in the process of evaluating OSS and commercial offerings.

System requirements at not particularly tight: - Memory running set of up to 10MB, - Can spare 20%+ of a 300MHz ARM and more in bursts, - UI will be in jQuery and JSON, so would like to feed several hundred KB pages linking a dozen CSS and JS files in well under a second.

Feature requirements: - HTTPS support, - A 10+ concurrent connections, - Well-tested against DOS attacks.

Would much appreciate an integrated XML parser to base a SOAP implementation on.

Not a fan of PHP, but not certain about server-side Javascript either, and unfamiliar with Lua. So looking for suggestions for templating solutions, perhaps a Python-based stack.

Already reviewed discussions on SO and lists on Wikipedia. Am aware of thttpd, Mongoose, Cherokee, Appweb.

At this point I invite detailed technical suggestions and discussion of implementation choices, based on first-hand experince in production quality deployment.

Answer 1

When it comes to a simple python server stack, the combination I've heard most often from within the community for lightweight implementation is CherryPy (to provide the thread-pooled WSGI server) with Werkzeug (to create the basic structure of the app) Both are very slightly different takes on WSGI that speed development time considerably.

There some pretty good notes outlining basic python framework comparisons (albeit not in an embedded environment, but the emphasis was on lightweight deployments.) at this question, in which Alex "the Machine" Martelli weighed in for these two.

If you can afford the overhead of the python interpreter (which I am assuming is ok as you included it in your eligible list), werkzeug is an excellent way to setup an app that consists of simple endpoints. Responses can be mimetyped inline to aid in outputing your UI libs (Jquery, etc). There are awesome examples on the Werkzeug docs.

One of the best resources I've been able to find on comparing WSGI servers (to satisfy your need for high concurrent connections and DOS survivability) can be found at Nicholas Piel's blog post on the subject, where CherryPy ranks in as one of the best "bang-for-your-buck" resources to speed-wise. The WSGI server in Cherry is deployment ready, and this can be used as the server process providing the environ to your Werkzeug app so you don't need to implement something heavier like Apache with mod_wsgi. Cherry is easily capable of an average around 2000 r/ps with response times well under a second while under moderate load.

Since I don't know what kind of device you will be deploying this on, I should mention of course that both of these platforms are regularly updated, so this too should be considered if for whatever reason allocating network resources to update the device is impractical.

By combining python's minidom module (v2.6+) with the endpoint routing in Werkzeug, you should also benefit from very good development speed. Constructing a complex url schema is simple using Werkzeug's Map feature, and the tutorial at their documentation page gives an awesome rundown on this. Between the two, it shouldn't be too difficult to get your web service up and running.

Answer 2

I presume that the purpose of your embedded web server is to provide an administrative interface for configuration, operations and status.

For disclosure, our company builds and deploys administrative web interfaces on many embedded systems with specs similar to what you describe based on our product, Web. You can find out more about our approach at http://uweb.workware.net.au/, and you can read a paper I presented at the Embedded Linux Conference 2010 at http://workware.net.au/papers/embedded-scripting.pdf which provides some detail of how we balance the size and performance concerns with rapid deployment through scripting.

You have two broad options. The first is to use a framework such as µWeb, or Barracuda server (mentioned above), or a an open source framework such as luci (http://luci.subsignal.org/trac). The second is to use a light-weight web server such as those you mention above and then build your own framework (presumably based on jQuery and JSON). The second option will take significantly longer and security is a concern as you address XSS, CRSF and DOS attacks.

In any event, I strongly suggest you stay away from PHP, Python or server-side Javascript. These are all too resource hungry for a 300MHz ARM platform.

Why the requirement for XML and SOAP if your admin UI is going to be jQuery and JSON? Do you have a separate requirement for SOAP support? If so, gSOAP is probably a reasonable choice (it's been a few years since I last used it).

Regarding https and 10+ concurrent sessions, note that the initial SSL handshake is significantly resource intensive on and embedded platform. If you are establishing new requests frequently (either because of new sessions, or because requests aren't pipelined), the platform will struggle. You can probably only establish 1-2 SSL connections/second.