How to scale Orion GE?
I have deployed an Orion instance in FILAB and I have configured the Cygnus inyector in order to store information in Cosmos.
But...let us imagine a scenario in which the number of entities increases drastically. In this hypothetical scenario one instance of Orion GE wouldn't be enough so it would be necessary to deploy more instances.
What would be the scale procedure? Taking into account the maximum quotas are:
VM Instances: 5 VCPUs: 10 Hard Disk: 100 GB Memory: 10240 MB Public IP: 1
I understand that quotas may be subject to changes but what would be the free account limit?
What would be the Hard disk limit in Cosmos Head Node? (Theoretically 5GB quota)
Would it be possible to deploy more instances of Orion Context Broker with a single public IP or would it be necessary to ask for multiple public ips? How?
To sum up, I request information about the scale procedure for the proposed scenario and the free account limits (maximum quotas possible).
Thank you in advance. Kind Regards.
Ramon.
Regarding Orion scalability, it could involve two dimensions:
Scalability in the number of entities. In this case, the scarce resource is the DB, so you would need to scale the MongoDB layer. The usual procedure to scale MongoDB is using shards, please check MongoDB official documentation abouit it.
Scalability in the operation requests to manage such entities. In this case, you can use additional Orion nodes (each one running in a separate VM, plus an additional VM in front of them running the load balancer software to distribute the load among Orion nodes). Orion is a stateless process that can run in such horizontal scaling configuration as long as:
1) you don't use ONTIMEINTERVAL subscriptions (see details in this post)(see UPDATE2 note below), 2) you have to configure the-subCacheIvalCLI parameter with a small enough value that ensures eventual consistency (basically, the value of the-subCacheIvalparameter is the maximum time that may pass from a subscriptions with entity patterns is done until it is consolidated in all the Orion nodes).
In any case, you would need additional VMs. You don't need additional IPs, as long as the system only needs a public IP (the one assigned to the load balancer) and all the other communications can be done internally. Cloud quota information has been already answered by @flopez in another post.
Ragarding the persistence of data in Cosmos through Cygnus, the same way you create a farm of Orion processes you may add more Cygnus processes in charge of receiving notifications from the Orion farm. Simply define a mapping strategy for all you entities, defining subscriptions about which entities are going to be notified to which Cygnus process A, which other to Cygnus process B, etc. The problem is on the connectivity between these Cygnus farm and the Global Instance of Cosmos (located in the Internet). Assuming these cygnus farm is running on top of VMs with private addressing, you must install some kind of proxy in another VM in order to access Cosmos.
About the HDFS quota, yes, it is 5 GB by default, but can be changed on demand. It worths saying a new HDFS cluster will be released in the short-term, having a higher capacity of storage.
UPDATE: a more detailed workflow explanation for the subscription-update-notification case is provided in this separated Q&A post.
UPDATE2: ONTIMEINTERVAL subscriptions were removed in Orion 1.0.0 (March 2016).
- Orion context broker doesn't create tenant databases "on the fly"
- Connect system which includes OPC UA IoT Agent, Orion Context Broker. Cygnus and Historic data with Postgres
- ERROR: Pool overlaps with other one on this address space
- Displaying Irrelevant Sensor Data in FIWARE Orion When Using IoT Agent-JSON and Mosquitto
- Get all Fiware-ServicePath for a specific Fiware-Service
- FIWARE PEP Proxy Wilma server fails to start
- FIWARE Orion-LD upsert replace endpoint: only attributes or completely (including type)?
- FIWARE FIROS Publish cmd_vel from Non-ROS-World
- IoTAgent (JSON) over MQTT doesn't receive measurements from mosquitto
- Orion sending error message "connection failed, after 100 retries"
- Orion-ld fails to establish a connection with the MongoDB Community Operator due to a connection string issue
- How to get the missing table Schema while using quantumleap in cratedb using orion LD context broker though the subscriptions are working?
- yum error "Cannot retrieve metalink for repository: epel. Please verify its path and try again" updating ContextBroker
- Specifying attributes to be aggregated by STHSink
- Why Orion-ld after 0.8.0 is not supporting the formation of service/subservice in quantumleap logs?
- issues configuring a device via bidirectional attributes using IOT agent JSON and Orion LD
- Error downloading and installing Java JDK in CentOS (FIWARE)
- Context provider application: MongoDB connection unsuccessful
- How to maintain a FIWARE apikey?
- How to get the servicePath from /v2/entities GET call
- Orion APIs authentication through Keycloak
- My dockerized service doesn't receive the notification sent by Orion Context Broker
- How to send massive data of sensors in Orion
- Notify python orion/quantumleap subscription changes
- Can I send a notification in the creation and deletion of an entity?
- HTTP POST request gets transformed to GET request
- Orion APIs authorization through Keycloak
- Fiware orion /op/query endpoint working explaination
- Subscription between Orion context broker and Quantumleap works, but table is not created in CrateDB
- Calling external APIs through fiware orion context broker to validate using keyrock