How to scale Orion GE?

Question

I have deployed an Orion instance in FILAB and I have configured the Cygnus inyector in order to store information in Cosmos.

But...let us imagine a scenario in which the number of entities increases drastically. In this hypothetical scenario one instance of Orion GE wouldn't be enough so it would be necessary to deploy more instances.

What would be the scale procedure? Taking into account the maximum quotas are:

VM Instances: 5 VCPUs: 10 Hard Disk: 100 GB Memory: 10240 MB Public IP: 1

I understand that quotas may be subject to changes but what would be the free account limit?

What would be the Hard disk limit in Cosmos Head Node? (Theoretically 5GB quota)

Would it be possible to deploy more instances of Orion Context Broker with a single public IP or would it be necessary to ask for multiple public ips? How?

To sum up, I request information about the scale procedure for the proposed scenario and the free account limits (maximum quotas possible).

Thank you in advance. Kind Regards.

Ramon.

Answer 1

Regarding Orion scalability, it could involve two dimensions:

• Scalability in the number of entities. In this case, the scarce resource is the DB, so you would need to scale the MongoDB layer. The usual procedure to scale MongoDB is using shards, please check MongoDB official documentation abouit it.

• Scalability in the operation requests to manage such entities. In this case, you can use additional Orion nodes (each one running in a separate VM, plus an additional VM in front of them running the load balancer software to distribute the load among Orion nodes). Orion is a stateless process that can run in such horizontal scaling configuration as long as: 1) you don't use ONTIMEINTERVAL subscriptions (see details in this post) (see UPDATE2 note below), 2) you have to configure the -subCacheIval CLI parameter with a small enough value that ensures eventual consistency (basically, the value of the -subCacheIval parameter is the maximum time that may pass from a subscriptions with entity patterns is done until it is consolidated in all the Orion nodes).

In any case, you would need additional VMs. You don't need additional IPs, as long as the system only needs a public IP (the one assigned to the load balancer) and all the other communications can be done internally. Cloud quota information has been already answered by @flopez in another post.

Ragarding the persistence of data in Cosmos through Cygnus, the same way you create a farm of Orion processes you may add more Cygnus processes in charge of receiving notifications from the Orion farm. Simply define a mapping strategy for all you entities, defining subscriptions about which entities are going to be notified to which Cygnus process A, which other to Cygnus process B, etc. The problem is on the connectivity between these Cygnus farm and the Global Instance of Cosmos (located in the Internet). Assuming these cygnus farm is running on top of VMs with private addressing, you must install some kind of proxy in another VM in order to access Cosmos.

About the HDFS quota, yes, it is 5 GB by default, but can be changed on demand. It worths saying a new HDFS cluster will be released in the short-term, having a higher capacity of storage.

UPDATE: a more detailed workflow explanation for the subscription-update-notification case is provided in this separated Q&A post.

UPDATE2: ONTIMEINTERVAL subscriptions were removed in Orion 1.0.0 (March 2016).