Apache Mesos Schedulers and Executors by example

Question

I am trying to understand how the various components of Mesos work together, and found this excellent tutorial that contains the following architectural overview:

I have a few concerns about this that aren't made clear (either in the article or in the official Mesos docs):

• Where are the Schedulers running? Are there "Scheduler nodes" where only the Schedulers should be running?
• If I was writing my own Mesos framework, what Scheduler functionality would I need to implement? Is it just a binary yes/no or accept/reject for Offers sent by the Master? Any concrete examples?
• If I was writing my own Mesos framework, what Executor functionality would I need to implement? Any concrete examples?
• What's a concrete example of a Task that would be sent to an Executor?
• Are Executors "pinned" (permanently installed on) Slaves, or do they float around in an "on demand" type fashion, being installed and executed dynamically/on-the-fly?

Answer 1

Great questions! I believe it would be really helpful to have a look at a sample framework such as Rendler. This will probably answer most of your question and give you feeling for the framework internal.

Let me now try to answer the question which might be still be open after this.

• Scheduler Location

Schedulers are not on on any special nodes, but keep in mind that schedulers can failover as well (as any part in a distributed system).

• Scheduler functionality

Have a look at Rendler or at the framework development guide.

• Executor functionality/Task

I believe Rendler is a good example to understand the Task/Executor relationship. Just start reading the README/description on the main github page.

• Executor pinning

Executors are started on each node when the first Task requiring such executor is send to this node. After this it will remain on that node.

Hope this helped!

Answer 2

To add to js84's excellent response,

• Scheduler Location: Many users like to launch the schedulers via another framework like Marathon to ensure that if the scheduler or its node dies, then it can be restarted elsewhere.
• Scheduler functionality: After registering with Mesos, your scheduler will start getting resource offers in the resourceOffers() callback, in which your scheduler should launch (at least) one task on a subset (or all) of the resources being offered. You'll probably also want to implement the statusUpdate() callback to handle task completion/failure. Note that you may not even need to implement your own scheduler if an existing framework like Marathon/Chronos/Aurora/Kubernetes could suffice.
• Executor functionality: You usually don't need to create a custom executor if you just want to launch a linux process or docker container and know when it completes. You could just use the default mesos-executor (by specifying a CommandInfo directly in TaskInfo, instead of embedded inside an ExecutorInfo). If, however you want to build a custom executor, at minimum you need to implement launchTask(), and ideally also killTask().
• Example Task: An example task could be a simple linux command like sleep 1000 or echo "Hello World", or a docker container (via ContainerInfo) like image : 'mysql'. Or, if you use a custom executor, then the executor defines what a task is and how to run it, so a task could instead be run as another thread in the executor's process, or just become an item in a queue in a single-threaded executor.
• Executor pinning: The executor is distributed via CommandInfo URIs, just like any task binaries, so they do not need to be preinstalled on the nodes. Mesos will fetch and run it for you.