I am working on a big project that involves a lot of web based and AI work. I am extremely comfortable with Python, though my only concern is with concurrent programming and scaling this project to make it work on clusters. Thus, Clojure for AI and support for Java function calls and bring about concurrent programming.
Is this a good idea to do all the web-based api work with Python and let Clojure take care of most of the concurrent AI work?
Edit: Let me explain the interaction in detail. Python would be doing most of the dirty work (scraping, image processing, improving the database and all that.) Clojure, if possible, would either deal with the data base or get the data from Python. I except something CPython sort of linking with Python and Clojure.
Edit2: Might be a foolish question to ask, but this being a rather long term project which will evolve quite a bit and go under several iterations, is Clojure a language here to stay? Is it portable enough?
Clojure is a dynamic, general-purpose programming language, combining the approachability and interactive development of a scripting language with an efficient and robust infrastructure for multithreaded programming.
Minimal syntax, very condensed and short API, no types. That simplicity, when comparing to other functional languages, makes it relatively easy to learn Clojure.
If you want to learn a language that smooths out the friction between thought processes and code, that is clear and understandable and where you're supported by a strong community, then Clojure is a great bet.
Clojure is stableLike Java, Clojure has a strong focus on API stability. Programs written for Clojure 1.0 in 2009 are still likely to work with Clojure 1.10. 1 in 2020.
I built an embarrassingly parallel number-crunching application with a backend in Clojure (on an arbitrary number of machines) and a frontend in Ruby on Rails. I don't particularly like RoR, but this was a zero-budget project at the time and we had a Rails programmer at hand who was willing to work for free.
The Clojure part consisted of (roughly) a controller, number crunching nodes, and a server implementing a JSON-over-HTTP API which was the interface to the Rails web app. The Clojure nodes used RabbitMQ to talk to each other. Because we defined clear APIs between different parts of the application, it was easy to later rewrite the frontend in Clojure (because that better suited our needs).
If you're working on a distributed project with a long life span and continuous development effort, it could make sense to design the application as a number of separate modules that communicate through well defined APIs (json, bson, ... over AMQP, HTTP, ... or a database). That means you can get started quickly using a language you're comfortable with, and rewrite parts in another language at a later stage if necessary.
I can't see a big problem with using Python for the web apps and Clojure for the concurrent data crunching / back end code. I assume you would use something like JSON over http for the communications between the two, which should work fine.
I'd personally use Clojure for both (using e.g. the excellent Noir as a web framework and Korma for the database stuff.), but if as you say your experience is mostly in Python then it probably makes sense to stick with Python from a productivity perspective (in the short term at least).
To answer the questions regarding the future of Clojure:
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With