I've just added an HP Aruba 2930F switch to our network. We use collectd to monitor all our switches, and all the others (Netgear, Dell) are working fine. But most stats are missing (never collected) from the Aruba.
For example, I can manually query ifinOctets using snmpbulkwalk and I can see many values returned using tcpdump (one for each port on the switch). It seems that snmpbulkwalk correctly iterates over the whole table. You can see it sending further GetBulk requests to fetch more values after the first response from the switch:
00:56:23.280128 IP 192.168.100.89.46401 > 10.24.1.69.snmp: C="public" GetBulk(29) N=0 M=6 .1.3.6.1.2.1.2.2.1.10
00:56:23.281528 IP 10.24.1.69.snmp > 192.168.100.89.46401: C="public" GetResponse(130) .1.3.6.1.2.1.2.2.1.10.1=2532749978 .1.3.6.1.2.1.2.2.1.10.2=1446933610 .1.3.6.1.2.1.2.2.1.10.3=2282826448 .1.3.6.1.2.1.2.2.1.10.4=243266224 .1.3.6.1.2.1.2.2.1.10.5=0 .1.3.6.1.2.1.2.2.1.10.6=0
00:56:23.281867 IP 192.168.100.89.46401 > 10.24.1.69.snmp: C="public" GetBulk(30) N=0 M=6 .1.3.6.1.2.1.2.2.1.10.6
00:56:23.283020 IP 10.24.1.69.snmp > 192.168.100.89.46401: C="public" GetResponse(129) .1.3.6.1.2.1.2.2.1.10.7=0 .1.3.6.1.2.1.2.2.1.10.8=0 .1.3.6.1.2.1.2.2.1.10.9=2946555352 .1.3.6.1.2.1.2.2.1.10.10=319508930 .1.3.6.1.2.1.2.2.1.10.11=40623969 .1.3.6.1.2.1.2.2.1.10.12=381333999
00:56:23.283256 IP 192.168.100.89.46401 > 10.24.1.69.snmp: C="public" GetBulk(30) N=0 M=6 .1.3.6.1.2.1.2.2.1.10.12
00:56:23.284371 IP 10.24.1.69.snmp > 192.168.100.89.46401: C="public" GetResponse(125) .1.3.6.1.2.1.2.2.1.10.13=1078444712 .1.3.6.1.2.1.2.2.1.10.14=749733074 1.3.6.1.2.1.2.2.1.10.15=0 .1.3.6.1.2.1.2.2.1.10.16=0 .1.3.6.1.2.1.2.2.1.10.17=0 .1.3.6.1.2.1.2.2.1.10.18=1126654495
00:56:23.284590 IP 192.168.100.89.46401 > 10.24.1.69.snmp: C="public" GetBulk(30) N=0 M=6 .1.3.6.1.2.1.2.2.1.10.18
00:56:23.285707 IP 10.24.1.69.snmp > 192.168.100.89.46401: C="public" GetResponse(116) .1.3.6.1.2.1.2.2.1.10.19=0 .1.3.6.1.2.1.2.2.1.10.20=0 .1.3.6.1.2.1.2.2.1.10.21=0 .1.3.6.1.2.1.2.2.1.10.22=0 .1.3.6.1.2.1.2.2.1.10.23=0 .1.3.6.1.2.1.2.2.1.10.24=0
00:56:23.285959 IP 192.168.100.89.46401 > 10.24.1.69.snmp: C="public" GetBulk(30) N=0 M=6 .1.3.6.1.2.1.2.2.1.10.24
00:56:23.287093 IP 10.24.1.69.snmp > 192.168.100.89.46401: C="public" GetResponse(127) .1.3.6.1.2.1.2.2.1.10.25=71003086 .1.3.6.1.2.1.2.2.1.10.26=0 .1.3.6.1.2.1.2.2.1.10.27=3740933620 .1.3.6.1.2.1.2.2.1.10.28=2646919069 .1.3.6.1.2.1.2.2.1.10.29=0 .1.3.6.1.2.1.2.2.1.10.30=0
00:56:23.287376 IP 192.168.100.89.46401 > 10.24.1.69.snmp: C="public" GetBulk(30) N=0 M=6 .1.3.6.1.2.1.2.2.1.10.30
00:56:23.288465 IP 10.24.1.69.snmp > 192.168.100.89.46401: C="public" GetResponse(126) .1.3.6.1.2.1.2.2.1.10.31=0 .1.3.6.1.2.1.2.2.1.10.32=52805387 .1.3.6.1.2.1.2.2.1.10.33=0 .1.3.6.1.2.1.2.2.1.10.34=2769493971 .1.3.6.1.2.1.2.2.1.10.35=968558097 .1.3.6.1.2.1.2.2.1.10.36=0
00:56:23.288690 IP 192.168.100.89.46401 > 10.24.1.69.snmp: C="public" GetBulk(30) N=0 M=6 .1.3.6.1.2.1.2.2.1.10.36
00:56:23.289798 IP 10.24.1.69.snmp > 192.168.100.89.46401: C="public" GetResponse(135) .1.3.6.1.2.1.2.2.1.10.37=513959840 .1.3.6.1.2.1.2.2.1.10.38=1121458074 1.3.6.1.2.1.2.2.1.10.39=1767286115 .1.3.6.1.2.1.2.2.1.10.40=552516006 .1.3.6.1.2.1.2.2.1.10.41=186221175 .1.3.6.1.2.1.2.2.1.10.42=4040939717
00:56:23.290010 IP 192.168.100.89.46401 > 10.24.1.69.snmp: C="public" GetBulk(30) N=0 M=6 .1.3.6.1.2.1.2.2.1.10.42
00:56:23.291064 IP 10.24.1.69.snmp > 192.168.100.89.46401: C="public" GetResponse(122) .1.3.6.1.2.1.2.2.1.10.43=1977099082 .1.3.6.1.2.1.2.2.1.10.44=0 .1.3.6.1.2.1.2.2.1.10.45=361247512 .1.3.6.1.2.1.2.2.1.10.46=0 .1.3.6.1.2.1.2.2.1.10.47=0 .1.3.6.1.2.1.2.2.1.10.48=0
00:56:23.291286 IP 192.168.100.89.46401 > 10.24.1.69.snmp: C="public" GetBulk(30) N=0 M=6 .1.3.6.1.2.1.2.2.1.10.48
00:56:23.292568 IP 10.24.1.69.snmp > 192.168.100.89.46401: C="public" GetResponse(121) .1.3.6.1.2.1.2.2.1.10.49=660138853 .1.3.6.1.2.1.2.2.1.10.50=0 .1.3.6.1.2.1.2.2.1.10.51=0 .1.3.6.1.2.1.2.2.1.10.52=0 .1.3.6.1.2.1.2.2.1.10.652=0 .1.3.6.1.2.1.2.2.1.10.654=0
00:56:23.292790 IP 192.168.100.89.46401 > 10.24.1.69.snmp: C="public" GetBulk(31) N=0 M=6 .1.3.6.1.2.1.2.2.1.10.654
00:56:23.293784 IP 10.24.1.69.snmp > 192.168.100.89.46401: C="public" GetResponse(122) .1.3.6.1.2.1.2.2.1.10.655=0 .1.3.6.1.2.1.2.2.1.10.656=0 .1.3.6.1.2.1.2.2.1.10.657=0 .1.3.6.1.2.1.2.2.1.10.658=0 .1.3.6.1.2.1.2.2.1.10.659=0 .1.3.6.1.2.1.2.2.1.10.660=0
However when collectd polls this switch, it seems to only request the first 6 values (fails to walk the whole table), and I don't know why:
01:04:44.915582 IP 192.168.100.89.51058 > 10.24.1.69.snmp: C="public" GetBulk(60) N=0 M=6 .1.3.6.1.2.1.2.2.1.10 .1.3.6.1.2.1.2.2.1.16 .1.3.6.1.2.1.31.1.1.1.1
01:04:44.917094 IP 10.24.1.69.snmp > 192.168.100.89.51058: C="public" GetResponse(366) .1.3.6.1.2.1.2.2.1.10.1=3583770206 .1.3.6.1.2.1.2.2.1.16.1=2607010378 .1.3.6.1.2.1.31.1.1.1.1.1="1" .1.3.6.1.2.1.2.2.1.10.1=3583770206 .1.3.6.1.2.1.2.2.1.16.1=2607010378 .1.3.6.1.2.1.31.1.1.1.1.2="2" .1.3.6.1.2.1.2.2.1.10.2=1451267024 .1.3.6.1.2.1.2.2.1.16.2=2517057927 .1.3.6.1.2.1.31.1.1.1.1.3="3" .1.3.6.1.2.1.2.2.1.10.3=2300985494 .1.3.6.1.2.1.2.2.1.16.3=3520776455 .1.3.6.1.2.1.31.1.1.1.1.4="4" .1.3.6.1.2.1.2.2.1.10.4=252143720 .1.3.6.1.2.1.2.2.1.16.4=3284598898 .1.3.6.1.2.1.31.1.1.1.1.5="5" .1.3.6.1.2.1.2.2.1.10.5=0 .1.3.6.1.2.1.2.2.1.16.5=0 .1.3.6.1.2.1.31.1.1.1.1.6="6"
(10 seconds pass, then it starts again)
01:04:54.915752 IP 192.168.100.89.51058 > 10.24.1.69.snmp: C="public" GetBulk(60) N=0 M=6 .1.3.6.1.2.1.2.2.1.10 .1.3.6.1.2.1.2.2.1.16 .1.3.6.1.2.1.31.1.1.1.1
01:04:54.917299 IP 10.24.1.69.snmp > 192.168.100.89.51058: C="public" GetResponse(366) .1.3.6.1.2.1.2.2.1.10.1=3597233925 .1.3.6.1.2.1.2.2.1.16.1=2608313099 .1.3.6.1.2.1.31.1.1.1.1.1="1" .1.3.6.1.2.1.2.2.1.10.1=3597233925 .1.3.6.1.2.1.2.2.1.16.1=2608313099 .1.3.6.1.2.1.31.1.1.1.1.2="2" .1.3.6.1.2.1.2.2.1.10.2=1451416135 .1.3.6.1.2.1.2.2.1.16.2=2517244769 .1.3.6.1.2.1.31.1.1.1.1.3="3" .1.3.6.1.2.1.2.2.1.10.3=2301234135 .1.3.6.1.2.1.2.2.1.16.3=3521261458 .1.3.6.1.2.1.31.1.1.1.1.4="4" .1.3.6.1.2.1.2.2.1.10.4=252280577 .1.3.6.1.2.1.2.2.1.16.4=3295333123 .1.3.6.1.2.1.31.1.1.1.1.5="5" .1.3.6.1.2.1.2.2.1.10.5=0 .1.3.6.1.2.1.2.2.1.16.5=0 .1.3.6.1.2.1.31.1.1.1.1.6="6"
When collectd polls other switches, it does correctly walk the whole table:
01:17:54.970366 IP 192.168.100.89.43339 > 10.24.1.72.snmp: C="public" GetBulk(60) N=0 M=6 .1.3.6.1.2.1.2.2.1.10 .1.3.6.1.2.1.2.2.1.16 .1.3.6.1.2.1.31.1.1.1.1
01:17:54.980427 IP 10.24.1.72.snmp > 192.168.100.89.43339: C="public" GetResponse(448) .1.3.6.1.2.1.2.2.1.10.1=1224986669 .1.3.6.1.2.1.2.2.1.16.1=1718890461 .1.3.6.1.2.1.31.1.1.1.1.1="GigabitEthernet1" .1.3.6.1.2.1.2.2.1.10.2=97027268 .1.3.6.1.2.1.2.2.1.16.2=113076172 .1.3.6.1.2.1.31.1.1.1.1.2="GigabitEthernet2" .1.3.6.1.2.1.2.2.1.10.3=549941524 .1.3.6.1.2.1.2.2.1.16.3=566368200 .1.3.6.1.2.1.31.1.1.1.1.3="GigabitEthernet3" .1.3.6.1.2.1.2.2.1.10.4=152280386 .1.3.6.1.2.1.2.2.1.16.4=1943593873 .1.3.6.1.2.1.31.1.1.1.1.4="GigabitEthernet4" .1.3.6.1.2.1.2.2.1.10.5=56531878 .1.3.6.1.2.1.2.2.1.16.5=237964204 .1.3.6.1.2.1.31.1.1.1.1.5="GigabitEthernet5" .1.3.6.1.2.1.2.2.1.10.6=0 .1.3.6.1.2.1.2.2.1.16.6=0 .1.3.6.1.2.1.31.1.1.1.1.6="GigabitEthernet6"
01:17:54.980541 IP 192.168.100.89.43339 > 10.24.1.72.snmp: C="public" GetBulk(63) N=0 M=6 .1.3.6.1.2.1.2.2.1.10.6 .1.3.6.1.2.1.2.2.1.16.6 .1.3.6.1.2.1.31.1.1.1.1.6
01:17:54.990723 IP 10.24.1.72.snmp > 192.168.100.89.43339: C="public" GetResponse(382) .1.3.6.1.2.1.2.2.1.10.7=0 .1.3.6.1.2.1.2.2.1.16.7=0 .1.3.6.1.2.1.31.1.1.1.1.7="GigabitEthernet7" .1.3.6.1.2.1.2.2.1.10.8=0 .1.3.6.1.2.1.2.2.1.16.8=0 .1.3.6.1.2.1.31.1.1.1.1.8="GigabitEthernet8" .1.3.6.1.2.1.2.2.1.10.1000=0 .1.3.6.1.2.1.2.2.1.16.1000=0 .1.3.6.1.2.1.31.1.1.1.1.1000="LAG1" .1.3.6.1.2.1.2.2.1.10.1001=0 .1.3.6.1.2.1.2.2.1.16.1001=0 .1.3.6.1.2.1.31.1.1.1.1.1001="LAG2" .1.3.6.1.2.1.2.2.1.10.1002=0 .1.3.6.1.2.1.2.2.1.16.1002=0 .1.3.6.1.2.1.31.1.1.1.1.1002="LAG3" .1.3.6.1.2.1.2.2.1.10.1003=0 .1.3.6.1.2.1.2.2.1.16.1003=0 .1.3.6.1.2.1.31.1.1.1.1.1003="LAG4"
01:17:54.990825 IP 192.168.100.89.43339 > 10.24.1.72.snmp: C="public" GetBulk(66) N=0 M=6 .1.3.6.1.2.1.2.2.1.10.1003 .1.3.6.1.2.1.2.2.1.16.1003 .1.3.6.1.2.1.31.1.1.1.1.1003
01:17:55.000842 IP 10.24.1.72.snmp > 192.168.100.89.43339: C="public" GetResponse(365) .1.3.6.1.2.1.2.2.1.10.1004=0 .1.3.6.1.2.1.2.2.1.16.1004=0 .1.3.6.1.2.1.31.1.1.1.1.1004="LAG5" .1.3.6.1.2.1.2.2.1.10.1005=0 .1.3.6.1.2.1.2.2.1.16.1005=0 .1.3.6.1.2.1.31.1.1.1.1.1005="LAG6" .1.3.6.1.2.1.2.2.1.10.1006=0 .1.3.6.1.2.1.2.2.1.16.1006=0 .1.3.6.1.2.1.31.1.1.1.1.1006="LAG7" .1.3.6.1.2.1.2.2.1.10.1007=0 .1.3.6.1.2.1.2.2.1.16.1007=0 .1.3.6.1.2.1.31.1.1.1.1.1007="LAG8" .1.3.6.1.2.1.2.2.1.10.10000=0 .1.3.6.1.2.1.2.2.1.16.10000=0 .1.3.6.1.2.1.31.1.1.1.1.10000="CPU" .1.3.6.1.2.1.2.2.1.11.1=45819844 .1.3.6.1.2.1.2.2.1.17.1=36740050 .1.3.6.1.2.1.31.1.1.1.2.1=6396023
I think there must be something that collectd doesn't like in the response for this particular switch, so it stops walking, but I'm not sure what. I guess it could be the repeated values in the response (in bold below), or the fact that the order of the OIDs in the response changes after these (which may or may not be allowed by the SNMP GetBulk spec, I'm not sure):
The relevant parts of collectd.conf are:
<Data "if_octets"> Type "if_octets" Table true Instance "IF-MIB::ifName" Values "ifInOctets" "ifOutOctets"
<Host "ch-mr-sw-02"> Address "10.24.1.69" Version 2 Community "public" Collect "uptime" "if_speed" "if_status" "if_octets" "if_errors" "if_dropped" "if_multicast" "if_broadcast" "if_uptime" Interval 10 BulkSize 20 Timeout 2 Retries 1
The manual says for the Data block's Values option:
If
Tableis set totrue, each OID must be the prefix of all the values to query, e. g.IF-MIB::ifInOctetsfor all the counters of incoming traffic. This subtree is walked (usingGETNEXT) until a value from outside the subtree is returned.
So it should be walking the whole subtree, but for just this one switch it doesn't.
You've pretty much answered your own question, but I'm answering to confirm what you've (mostly) already guessed.
The behavior of this switch is just plain wrong. The part of the packet capture that says GetBulk(60) N=0 M=6 tells you that the non-repeaters value is 0 and the max-repetitions is 6. The response is supposed to give you the next object in order for each of your three OIDs, and then the next object in order for each of those three OIDs, and so on, up to 6 repetitions. What you're seeing is not a changing of the order, it's just that the first two OIDs in the second set of three are not advancing properly, and for some reason the third one is, and then the later groupings of three propagate that error:
.1.3.6.1.2.1.2.2.1.10.1=3583770206 .1.3.6.1.2.1.2.2.1.16.1=2607010378 .1.3.6.1.2.1.31.1.1.1.1.1="1"
.1.3.6.1.2.1.2.2.1.10.1=3583770206 .1.3.6.1.2.1.2.2.1.16.1=2607010378 .1.3.6.1.2.1.31.1.1.1.1.2="2"
.1.3.6.1.2.1.2.2.1.10.2=1451267024 .1.3.6.1.2.1.2.2.1.16.2=2517057927 .1.3.6.1.2.1.31.1.1.1.1.3="3"
.1.3.6.1.2.1.2.2.1.10.3=2300985494 .1.3.6.1.2.1.2.2.1.16.3=3520776455 .1.3.6.1.2.1.31.1.1.1.1.4="4"
.1.3.6.1.2.1.2.2.1.10.4=252143720 .1.3.6.1.2.1.2.2.1.16.4=3284598898 .1.3.6.1.2.1.31.1.1.1.1.5="5"
.1.3.6.1.2.1.2.2.1.10.5=0 .1.3.6.1.2.1.2.2.1.16.5=0 .1.3.6.1.2.1.31.1.1.1.1.6="6"
I see that your config has an "interval" of 10. I suspect that it just doesn't know how to handle this kind of misbehavior, so it aborts until the next interval.