What libraries should I use for better OCaml Threading?

Question

I have asked a related question before Why OCaml's threading is considered as `not enough`?

No matter how "bad" ocaml's threading is, I notice some libraries say they can do real threading.

For example, Lwt

Lwt offers a new alternative. It provides very light-weight cooperative threads; ``launching'' a thread is a very fast operation, it does not require a new stack, a new process, or anything else. Moreover context switches are very fast. In fact, it is so easy that we will launch a thread for every system call. And composing cooperative threads will allow us to write highly asynchronous programs.

Also Jane Street's aync_core also provides similar things, if I am right.

---

But I am quite confused. Do Lwt or aync_core provide threading like Java threading?

If I use them, can I utilise multiple cpu?

In what way, can I get a "real threading" (just like in Java) in OCaml?

---

Edit

I am still confused.

Let me add a scenario:

I have a server (16 cpu cores) and a server application.

What the server application does are:

• It listens to requests
• For each request, it starts a computational task (let's say costs 2 minutes to finish)
• When each task finishes, the task will either return the result back to the main or just send the result back to client directly

---

In Java, it is very easy. I create a thread pool, then for each request, I create a thread in that pool. that thread will run the computational task. This is mature in Java and it can utilize the 16 cpu cores. Am I right?

So my question is: can I do the same thing in OCaml?

Answer 1

The example of parallelized server that you cite is one of those embarassingly parallel problem that are well solved with a simple multiprocessing model, using fork. This has been doable in OCaml for decades, and yes, you will an almost linear speedup using all the cores of your machine if you need.

To do that using the simple primitives of the standard library, see this Chapter of the online book "Unix system programming in OCaml" (first released in 2003), and/or this chapter of the online book "Developing Applications with OCaml" (first released in 2000).

You may also want to use higher-level libraries such as Gerd Stolpmann's OCamlnet library mentioned by rafix, which provides a lot of stuff from direct helper for the usual client/server design, to lower-level multiprocess communication libraries; see the documentation.

The library Parmap is also interesting, but maybe for slightly different use case (it's more that you have a large array of data available all at the same time, that you want to process with the same function in parallel): a drop-in remplacement of Array.map or List.map (or fold) that parallelizes computations.