rulescephproxmoxcrush

Need help to setup a crush rule in ceph for ssd and hdd osd


we are having a proxmox cluster with 3 nodes. Each node have 4 ssd and 12 hdd. My plan is to create 2 crush rules (one for ssd devices and another one for hdd devices). With these 2 rules I will create 2 pools. One ssd pool and one hdd pool.

But inside the ceph documentation I found this https://docs.ceph.com/en/latest/rados/operations/crush-map/#custom-crush-rules. I am trying to understand this rule. Would this rule be more useful for my hardware? Can somebody explain (with simple words), what this rule is doing?

Thank you so much.


Solution

  • The easiest way to use SSDs or HDDs in your crush rules would be these, assuming you're using replicated pools:

    rule rule_ssd {
            id 1
            type replicated
            min_size 1
            max_size 10
            step take default class ssd
            step chooseleaf firstn 0 type host
            step emit
    }
    rule rule_hdd {
            id 2
            type replicated
            min_size 1
            max_size 10
            step take default class hdd
            step chooseleaf firstn 0 type host
            step emit
    }
    

    These rules make sure to select the desired device class (ssd or hdd) and choose any host within that selection, depending on your pool size (don't use size=2 except for testing purposes) it will choose that many hosts. So in this case the failure-domain is "host". The rule you refer to in the docs has its purpose in the name "mixed_replicated_rule". It spreads the replicas across different device classes (by the way, the autoscaler doesn't work well with mixed devices classes), I wouldn't really recommend it unless you have a good reason to. Stick to the easy ruleset and just use devices classes which usually are automatically detected when adding the drives.