Should I use multiple knowledge bases?

Question

We have a system that will have tens of thousands of units. Each unit will have 5-10 meters and each meter will have a value associated with it. The values of these meters change and we need our rules engine to be able to respond to these changes in realtime.

We will have rules of the sort "if the first meter from unit #1 is greater than 10 and the second meter from unit #1 is less than 30 then ...", although the rules may get much longer than this. The rules for each unit will be completely independent, so there will be no rules that condition on the values of two different meters from two different units.

We'll have about 30 rules that are the same for every unit, and then each unit will have about 5-15 custom rules. These rules will need to be added dynamically while the rules engine is running. It's likely a unit will add 5-10 rules right when it signs up and then add or a remove a rule something like once a week from that point on.

We decided to use Drools for this and I'm trying to figure out how best to implement it. I'm really new to Drools so I'm kinda confused. Would it make sense for each unit to have its own knowledge base? If so, is there any way to share the rules that are the same for each unit?

I'm worried we might not have enough memory to store all these rules, so I was thinking if we had a knowledge base for each unit, we could just serialize all the knowledge bases, put them in a database, and retrieve them when we need them. Would that be reasonable?

The other reason I was thinking of using a separate knowledge base for each unit is because each unit's rules are completely independent from every other unit's rules, there might be a performance hit from putting them all into the same knowledge base. Is this correct or is the Rete algorithm smart enough to figure that out?

Also, is dynamically adding rules while the engine is running possible? Will all the rules have to be recompiled? How much time will it take and is this feasible if the engine still needs to be responding to meter changes in realtime?

Thanks guys.

Answer 1

Would it make sense for each unit to have its own knowledge base? If so, is there any way to share the rules that are the same for each unit?

See below on each unit to have its own knowledge base. For organization purpose, you can think of putting them in separate packages. To share rules between packages, you can create the rules in global area and import them. Though this will help share the common rules, it has some flaws:

• When you have a large number of packages, you will need to manually import them first time.
• When you modify the imported rule in a specific package, it will change the global rule and so the change affects every other package which has the rule imported. In guvnor UI there is no way to visibly tell if a rule is imported or specific to package.

I'm worried we might not have enough memory to store all these rules, so I was thinking if we had a knowledge base for each unit, we could just serialize all the knowledge bases, put them in a database, and retrieve them when we need them. Would that be reasonable?

This is an option but if the rules change you will need recreate the knowledge bases.

The other reason I was thinking of using a separate knowledge base for each unit is because each unit's rules are completely independent from every other unit's rules, there might be a performance hit from putting them all into the same knowledge base. Is this correct or is the Rete algorithm smart enough to figure that out?

The book Drools JBoss Rules 5.0 Developer's Guide by Michal Bali says

The performance of the Rete algorithm is theoretically independent of the number of rules in the knowledge base.

Further,

If you are curious and want to see some benchmarks, you can find them in the drools-examples module that is downloadable from the Drools web site. There are also some web sites that regularly publish benchmarks of various rule engines solving well-known mathematical problems (usually 'Miss Manners' test and the 'Waltz'), for example, http://illation.com.au/ benchmarks/. Performance of Drools is comparable to other open source or even commercial engines.

Also, is dynamically adding rules while the engine is running possible? Will all the rules have to be recompiled? How much time will it take and is this feasible if the engine still needs to be responding to meter changes in realtime?

No concrete idea here but I am sure the rules need to be recompiled to be useable. The previously created kbase can still be used when the new packages are being recompiled. They are independent.