Rules can be used to make your configuration more dynamic, allowing values to change depending on the time or the value of a node attribute. Examples of things rules are useful for:
Each constraint type or property set that supports rules may contain one or more rule elements specifying conditions under which the constraint or properties take effect. Examples later in this chapter will make this clearer.
| Attribute | Default | Description |
|---|---|---|
| id | A unique name for this element (required) |
|
| role | Started | The rule is in effect only when the resource is in the specified role. Allowed values are Started, Unpromoted, and Promoted. A rule with a role of Promoted cannot determine the initial location of a clone instance and will only affect which of the active instances will be promoted. |
| score | If this rule is used in a location constraint and evaluates to true, apply this score to the constraint. Only one of score and score-attribute may be used. |
|
| score-attribute | If this rule is used in a location constraint and evaluates to true, use the value of this node attribute as the score to apply to the constraint. Only one of score and score-attribute may be used. |
|
| boolean-op | and | If this rule contains more than one condition, a value of and specifies that the rule evaluates to true only if all conditions are true, and a value of or specifies that the rule evaluates to true if any condition is true. |
A rule element must contain one or more conditions. A condition may be an expression element, a date_expression element, or another rule element.
Expressions are rule conditions based on the values of node attributes.
| Attribute | Default | Description |
|---|---|---|
| id | A unique name for this element (required) |
|
| attribute | The node attribute to test (required) |
|
| type | The default type for lt, gt, lte, and gte operations is number if either value contains a decimal point character, or integer otherwise. The default type for all other operations is string. If a numeric parse fails for either value, then the values are compared as type string. | How the node attributes should be compared. Allowed values are string, integer (since 2.0.5), number, and version. integer truncates floating-point values if necessary before performing a 64-bit integer comparison. number performs a double-precision floating-point comparison (32-bit integer before 2.0.5). |
| operation | The comparison to perform (required). Allowed values:
|
|
| value | User-supplied value for comparison (required for operations other than defined and not_defined) |
|
| value-source | literal | How the value is derived. Allowed values:
|
In addition to custom node attributes defined by the administrator, the cluster defines special, built-in node attributes for each node that can also be used in rule expressions.
| Name | Value |
|---|---|
| #uname | Node name |
| #id | Node ID |
| #kind | Node type. Possible values are cluster, remote, and container. Kind is remote for Pacemaker Remote nodes created with the ocf:pacemaker:remote resource, and container for Pacemaker Remote guest nodes and bundle nodes |
| #is_dc | true if this node is the cluster’s Designated Controller (DC), false otherwise |
| #cluster-name | The value of the cluster-name cluster property, if set |
| #site-name | The value of the site-name node attribute, if set, otherwise identical to #cluster-name |
| #role | The role the relevant promotable clone resource has on this node. Valid only within a rule for a location constraint for a promotable clone resource. |
Date/time expressions are rule conditions based (as the name suggests) on the current date and time.
A date_expression element may optionally contain a date_spec or duration element depending on the context.
| Attribute | Description |
|---|---|
| id | A unique name for this element (required) |
| start | A date/time conforming to the ISO8601 [https://en.wikipedia.org/wiki/ISO_8601] specification. May be used when operation is in_range (in which case at least one of start or end must be specified) or gt (in which case start is required). |
| end | A date/time conforming to the ISO8601 [https://en.wikipedia.org/wiki/ISO_8601] specification. May be used when operation is in_range (in which case at least one of start or end must be specified) or lt (in which case end is required). |
| operation | Compares the current date/time with the start and/or end date, depending on the context. Allowed values:
|
Note
There is no eq, neq, gte, or lte operation, since they would be valid only for a single second.
A date_spec element is used to create a cron-like expression relating to time. Each field can contain a single number or range. Any field not supplied is ignored.
| Attribute | Description |
|---|---|
| id | A unique name for this element (required) |
| hours | Allowed values: 0-23 (where 0 is midnight and 23 is 11 p.m.) |
| monthdays | Allowed values: 1-31 (depending on month and year) |
| weekdays | Allowed values: 1-7 (where 1 is Monday and 7 is Sunday) |
| yeardays | Allowed values: 1-366 (depending on the year) |
| months | Allowed values: 1-12 |
| weeks | Allowed values: 1-53 (depending on weekyear) |
| years | Year according to the Gregorian calendar |
| weekyears | Year in which the week started; for example, 1 January 2005 can be specified in ISO 8601 as “2005-001 Ordinal”, “2005-01-01 Gregorian” or “2004-W53-6 Weekly” and thus would match years="2005" or weekyears="2004" |
| moon | Allowed values are 0-7 (where 0 is the new moon and 4 is full moon). Seriously, you can use this. This was implemented to demonstrate the ease with which new comparisons could be added. |
For example, monthdays="1" matches the first day of every month, and hours="09-17" matches the hours between 9 a.m. and 5 p.m. (inclusive).
At this time, multiple ranges (e.g. weekdays="1,2" or weekdays="1-2,5-6") are not supported.
Note
Pacemaker can calculate when evaluation of a date_expression with an operation of gt, lt, or in_range will next change, and schedule a cluster re-check for that time. However, it does not do this for date_spec. Instead, it evaluates the date_spec whenever a cluster re-check naturally happens via a cluster event or the cluster-recheck-interval cluster option.
For example, if you have a date_spec enabling a resource from 9 a.m. to 5 p.m., and cluster-recheck-interval has been set to 5 minutes, then sometime between 9 a.m. and 9:05 a.m. the cluster would notice that it needs to start the resource, and sometime between 5 p.m. and 5:05 p.m. it would realize that it needs to stop the resource. The timing of the actual start and stop actions will further depend on factors such as any other actions the cluster may need to perform first, and the load of the machine.
A duration is used to calculate a value for end when one is not supplied to in_range operations. It contains one or more attributes each containing a single number. Any attribute not supplied is ignored.
| Attribute | Description |
|---|---|
| id | A unique name for this element (required) |
| seconds | This many seconds will be added to the total duration |
| minutes | This many minutes will be added to the total duration |
| hours | This many hours will be added to the total duration |
| weeks | This many weeks will be added to the total duration |
| months | This many months will be added to the total duration |
| years | This many years will be added to the total duration |
A small sample of how time-based expressions can be used:
True if now is any time in the year 2005
<rule id="rule1" score="INFINITY">
<date_expression id="date_expr1" start="2005-001" operation="in_range">
<duration id="duration1" years="1"/>
</date_expression>
</rule>
or equivalently:
<rule id="rule2" score="INFINITY">
<date_expression id="date_expr2" operation="date_spec">
<date_spec id="date_spec2" years="2005"/>
</date_expression>
</rule>
9 a.m. to 5 p.m. Monday through Friday
<rule id="rule3" score="INFINITY">
<date_expression id="date_expr3" operation="date_spec">
<date_spec id="date_spec3" hours="9-16" weekdays="1-5"/>
</date_expression>
</rule>
Note that the 16 matches all the way through 16:59:59, because the numeric value of the hour still matches.
9 a.m. to 6 p.m. Monday through Friday or anytime Saturday
<rule id="rule4" score="INFINITY" boolean-op="or">
<date_expression id="date_expr4-1" operation="date_spec">
<date_spec id="date_spec4-1" hours="9-16" weekdays="1-5"/>
</date_expression>
<date_expression id="date_expr4-2" operation="date_spec">
<date_spec id="date_spec4-2" weekdays="6"/>
</date_expression>
</rule>
9 a.m. to 5 p.m. or 9 p.m. to 12 a.m. Monday through Friday
<rule id="rule5" score="INFINITY" boolean-op="and">
<rule id="rule5-nested1" score="INFINITY" boolean-op="or">
<date_expression id="date_expr5-1" operation="date_spec">
<date_spec id="date_spec5-1" hours="9-16"/>
</date_expression>
<date_expression id="date_expr5-2" operation="date_spec">
<date_spec id="date_spec5-2" hours="21-23"/>
</date_expression>
</rule>
<date_expression id="date_expr5-3" operation="date_spec">
<date_spec id="date_spec5-3" weekdays="1-5"/>
</date_expression>
</rule>
Mondays in March 2005
<rule id="rule6" score="INFINITY" boolean-op="and">
<date_expression id="date_expr6-1" operation="date_spec">
<date_spec id="date_spec6" weekdays="1"/>
</date_expression>
<date_expression id="date_expr6-2" operation="in_range"
start="2005-03-01" end="2005-04-01"/>
</rule>
Note
Because no time is specified with the above dates, 00:00:00 is implied. This means that the range includes all of 2005-03-01 but none of 2005-04-01. You may wish to write end as "2005-03-31T23:59:59" to avoid confusion.
A full moon on Friday the 13th
<rule id="rule7" score="INFINITY" boolean-op="and">
<date_expression id="date_expr7" operation="date_spec">
<date_spec id="date_spec7" weekdays="5" monthdays="13" moon="4"/>
</date_expression>
</rule>
An rsc_expression (since 2.0.5) is a rule condition based on a resource agent’s properties. This rule is only valid within an rsc_defaults or op_defaults context. None of the matching attributes of class, provider, and type are required. If one is omitted, all values of that attribute will match. For instance, omitting type means every type will match.
| Attribute | Description |
|---|---|
| id | A unique name for this element (required) |
| class | The standard name to be matched against resource agents |
| provider | If given, the vendor to be matched against resource agents (only relevant when class is ocf) |
| type | The name of the resource agent to be matched |
A small sample of how resource-based expressions can be used:
True for all ocf:heartbeat:IPaddr2 resources
<rule id="rule1" score="INFINITY">
<rsc_expression id="rule_expr1" class="ocf" provider="heartbeat" type="IPaddr2"/>
</rule>
Provider doesn’t apply to non-OCF resources
<rule id="rule2" score="INFINITY">
<rsc_expression id="rule_expr2" class="stonith" type="fence_xvm"/>
</rule>
An op_expression (since 2.0.5) is a rule condition based on an action of some resource agent. This rule is only valid within an op_defaults context.
| Attribute | Description |
|---|---|
| id | A unique name for this element (required) |
| name | The action name to match against. This can be any action supported by the resource agent; common values include monitor, start, and stop (required). |
| interval | The interval of the action to match against. If not given, only the name attribute will be used to match. |
A small sample of how operation-based expressions can be used:
True for all monitor actions
<rule id="rule1" score="INFINITY">
<op_expression id="rule_expr1" name="monitor"/>
</rule>
True for all monitor actions with a 10 second interval
<rule id="rule2" score="INFINITY">
<op_expression id="rule_expr2" name="monitor" interval="10s"/>
</rule>
A location constraint may contain one or more top-level rules. The cluster will act as if there is a separate location constraint for each rule that evaluates as true.
Consider the following simple location constraint:
Prevent resource webserver from running on node node3
<rsc_location id="ban-apache-on-node3" rsc="webserver"
score="-INFINITY" node="node3"/>
The same constraint can be more verbosely written using a rule:
Prevent resource webserver from running on node node3 using a rule
<rsc_location id="ban-apache-on-node3" rsc="webserver">
<rule id="ban-apache-rule" score="-INFINITY">
<expression id="ban-apache-expr" attribute="#uname"
operation="eq" value="node3"/>
</rule>
</rsc_location>
The advantage of using the expanded form is that one could add more expressions (for example, limiting the constraint to certain days of the week), or activate the constraint by some node attribute other than node name.
The expanded form allows us to match on node properties other than its name. If we rated each machine’s CPU power such that the cluster had the following nodes section:
Sample node section with node attributes
<nodes>
<node id="uuid1" uname="c001n01" type="normal">
<instance_attributes id="uuid1-custom_attrs">
<nvpair id="uuid1-cpu_mips" name="cpu_mips" value="1234"/>
</instance_attributes>
</node>
<node id="uuid2" uname="c001n02" type="normal">
<instance_attributes id="uuid2-custom_attrs">
<nvpair id="uuid2-cpu_mips" name="cpu_mips" value="5678"/>
</instance_attributes>
</node>
</nodes>
then we could prevent resources from running on underpowered machines with this rule:
Rule using a node attribute (to be used inside a location constraint)
<rule id="need-more-power-rule" score="-INFINITY">
<expression id="need-more-power-expr" attribute="cpu_mips"
operation="lt" value="3000"/>
</rule>
When using score-attribute instead of score, each node matched by the rule has its score adjusted differently, according to its value for the named node attribute. Thus, in the previous example, if a rule inside a location constraint for a resource used score-attribute="cpu_mips", c001n01 would have its preference to run the resource increased by 1234 whereas c001n02 would have its preference increased by 5678.
Rules may be used to control a variety of options:
Note
Attribute-based expressions for meta-attributes can only be used within operations and op_defaults. They will not work with resource configuration or rsc_defaults. Additionally, attribute-based expressions cannot be used with cluster options.
Often some cluster nodes will be different from their peers. Sometimes, these differences – e.g. the location of a binary or the names of network interfaces – require resources to be configured differently depending on the machine they’re hosted on.
By defining multiple instance_attributes objects for the resource and adding a rule to each, we can easily handle these special cases.
In the example below, mySpecialRsc will use eth1 and port 9999 when run on node1, eth2 and port 8888 on node2 and default to eth0 and port 9999 for all other nodes.
Defining different resource options based on the node name
<primitive id="mySpecialRsc" class="ocf" type="Special" provider="me">
<instance_attributes id="special-node1" score="3">
<rule id="node1-special-case" score="INFINITY" >
<expression id="node1-special-case-expr" attribute="#uname"
operation="eq" value="node1"/>
</rule>
<nvpair id="node1-interface" name="interface" value="eth1"/>
</instance_attributes>
<instance_attributes id="special-node2" score="2" >
<rule id="node2-special-case" score="INFINITY">
<expression id="node2-special-case-expr" attribute="#uname"
operation="eq" value="node2"/>
</rule>
<nvpair id="node2-interface" name="interface" value="eth2"/>
<nvpair id="node2-port" name="port" value="8888"/>
</instance_attributes>
<instance_attributes id="defaults" score="1" >
<nvpair id="default-interface" name="interface" value="eth0"/>
<nvpair id="default-port" name="port" value="9999"/>
</instance_attributes>
</primitive>
The order in which instance_attributes objects are evaluated is determined by their score (highest to lowest). If not supplied, the score defaults to zero. Objects with an equal score are processed in their listed order. If the instance_attributes object has no rule, or a rule that evaluates to true, then for any parameter the resource does not yet have a value for, the resource will use the parameter values defined by the instance_attributes.
For example, given the configuration above, if the resource is placed on node1:
Rules can be used for resource and operation defaults. The following example illustrates how to set a different resource-stickiness value during and outside work hours. This allows resources to automatically move back to their most preferred hosts, but at a time that (in theory) does not interfere with business activities.
Change resource-stickiness during working hours
<rsc_defaults>
<meta_attributes id="core-hours" score="2">
<rule id="core-hour-rule" score="0">
<date_expression id="nine-to-five-Mon-to-Fri" operation="date_spec">
<date_spec id="nine-to-five-Mon-to-Fri-spec" hours="9-16" weekdays="1-5"/>
</date_expression>
</rule>
<nvpair id="core-stickiness" name="resource-stickiness" value="INFINITY"/>
</meta_attributes>
<meta_attributes id="after-hours" score="1" >
<nvpair id="after-stickiness" name="resource-stickiness" value="0"/>
</meta_attributes>
</rsc_defaults>
Rules may be used similarly in instance_attributes or utilization blocks.
Any single block may directly contain only a single rule, but that rule may itself contain any number of rules.
rsc_expression and op_expression blocks may additionally be used to set defaults on either a single resource or across an entire class of resources with a single rule. rsc_expression may be used to select resource agents within both rsc_defaults and op_defaults, while op_expression may only be used within op_defaults. If multiple rules succeed for a given resource agent, the last one specified will be the one that takes effect. As with any other rule, boolean operations may be used to make more complicated expressions.
Default all IPaddr2 resources to stopped
<rsc_defaults>
<meta_attributes id="op-target-role">
<rule id="op-target-role-rule" score="INFINITY">
<rsc_expression id="op-target-role-expr" class="ocf" provider="heartbeat"
type="IPaddr2"/>
</rule>
<nvpair id="op-target-role-nvpair" name="target-role" value="Stopped"/>
</meta_attributes>
</rsc_defaults>
Default all monitor action timeouts to 7 seconds
<op_defaults>
<meta_attributes id="op-monitor-defaults">
<rule id="op-monitor-default-rule" score="INFINITY">
<op_expression id="op-monitor-default-expr" name="monitor"/>
</rule>
<nvpair id="op-monitor-timeout" name="timeout" value="7s"/>
</meta_attributes>
</op_defaults>
Default the timeout on all 10-second-interval monitor actions on IPaddr2 resources to 8 seconds
<op_defaults>
<meta_attributes id="op-monitor-and">
<rule id="op-monitor-and-rule" score="INFINITY">
<rsc_expression id="op-monitor-and-rsc-expr" class="ocf" provider="heartbeat"
type="IPaddr2"/>
<op_expression id="op-monitor-and-op-expr" name="monitor" interval="10s"/>
</rule>
<nvpair id="op-monitor-and-timeout" name="timeout" value="8s"/>
</meta_attributes>
</op_defaults>
Controlling cluster options is achieved in much the same manner as specifying different resource options on different nodes.
The following example illustrates how to set maintenance_mode during a scheduled maintenance window. This will keep the cluster running but not monitor, start, or stop resources during this time.
Schedule a maintenance window for 9 to 11 p.m. CDT Sept. 20, 2019
<crm_config>
<cluster_property_set id="cib-bootstrap-options">
<nvpair id="bootstrap-stonith-enabled" name="stonith-enabled" value="1"/>
</cluster_property_set>
<cluster_property_set id="normal-set" score="10">
<nvpair id="normal-maintenance-mode" name="maintenance-mode" value="false"/>
</cluster_property_set>
<cluster_property_set id="maintenance-window-set" score="1000">
<nvpair id="maintenance-nvpair1" name="maintenance-mode" value="true"/>
<rule id="maintenance-rule1" score="INFINITY">
<date_expression id="maintenance-date1" operation="in_range"
start="2019-09-20 21:00:00 -05:00" end="2019-09-20 23:00:00 -05:00"/>
</rule>
</cluster_property_set>
</crm_config>
Important
The cluster_property_set with an id set to “cib-bootstrap-options” will always have the highest priority, regardless of any scores. Therefore, rules in another cluster_property_set can never take effect for any properties listed in the bootstrap set.