alba purge-osd : decommissioned

[yongshengma]

Hello,

I'm pretty sure the following device is not in use and missing from alba plugin GUI.

In [31]: [ c for c in claimed_osds if c['ips'][0]==u'192.168.20.33' and c['id']==0 ]
Out[31]: 
[{u'alba_id': u'9d7bdd52-b704-465b-9253-7156e1bf088d',
  u'albamgr_cfg': None,
  u'checksum_errors': 0,
  u'claimed_since': 1543828700.586121,
  u'decommissioned': True,
  u'endpoints': None,
  u'errors': [[1563349242.58375,
    u'Asd_client.BadLongId("wKpwkiXci7jHuDfAfFwnLCploMijii1e", "J98Ei4HdWR6L7oaJqq9TfjuqLrrZL7p8")'],
   [1563349239.129,
    u'Asd_client.BadLongId("wKpwkiXci7jHuDfAfFwnLCploMijii1e", "J98Ei4HdWR6L7oaJqq9TfjuqLrrZL7p8")'],
   [1563349234.427287,
    u'Asd_client.BadLongId("wKpwkiXci7jHuDfAfFwnLCploMijii1e", "J98Ei4HdWR6L7oaJqq9TfjuqLrrZL7p8")'],
   [1563349185.722885,
    u'Asd_client.BadLongId("wKpwkiXci7jHuDfAfFwnLCploMijii1e", "J98Ei4HdWR6L7oaJqq9TfjuqLrrZL7p8")'],
   [1563349179.758825,
    u'Asd_client.BadLongId("wKpwkiXci7jHuDfAfFwnLCploMijii1e", "J98Ei4HdWR6L7oaJqq9TfjuqLrrZL7p8")'],
   [1563349176.255778,
    u'Asd_client.BadLongId("wKpwkiXci7jHuDfAfFwnLCploMijii1e", "J98Ei4HdWR6L7oaJqq9TfjuqLrrZL7p8")'],
   [1563349130.081002,
    u'Asd_client.BadLongId("wKpwkiXci7jHuDfAfFwnLCploMijii1e", "J98Ei4HdWR6L7oaJqq9TfjuqLrrZL7p8")'],
   [1563349113.89867,
    u'Asd_client.BadLongId("wKpwkiXci7jHuDfAfFwnLCploMijii1e", "J98Ei4HdWR6L7oaJqq9TfjuqLrrZL7p8")'],
   [1563349113.796267,
    u'Asd_client.BadLongId("wKpwkiXci7jHuDfAfFwnLCploMijii1e", "J98Ei4HdWR6L7oaJqq9TfjuqLrrZL7p8")'],
   [1563349110.6479,
    u'Asd_client.BadLongId("wKpwkiXci7jHuDfAfFwnLCploMijii1e", "J98Ei4HdWR6L7oaJqq9TfjuqLrrZL7p8")']],
  u'id': 0,
  u'ips': [u'192.168.20.33'],
  u'kind': u'AsdV1',
  u'long_id': u'wKpwkiXci7jHuDfAfFwnLCploMijii1e',
  u'node_id': u'MtRUTNalzCedpLjg',
  u'port': 8601,
  u'prefix': None,
  u'preset': None,
  u'read': [1563102254.102759,
   1563102253.740176,
   1563102253.592432,
   1563102230.184313,
   1563102225.22527,
   1563102225.157166,
   1563102200.238294,
   1563102200.157486,
   1563102175.467823,
   1563102169.76271],
  u'seen': [1543828696.673785],
  u'total': 1199613935616,
  u'use_rdma': False,
  u'use_tls': False,
  u'used': 516273112914,
  u'write': [1563102284.511966,
   1563102284.003097,
   1563102283.920963,
   1563102283.790112,
   1563102283.721009,
   1563102283.24091,
   1563102283.184723,
   1563102283.171877,
   1563102283.153193,
   1563102282.971391]}]

So I purged it

[root@node1 ~]# alba purge-osd --long-id wKpwkiXci7jHuDfAfFwnLCploMijii1e --config arakoon://config/ovs/arakoon/sas-back-abm/config?ini=%2Fopt%OpenvStorage%2Fconfig%2Farakoon_cacc.ini
2019-07-17 15:27:06 513210 +0800 - node1 - 22276/0000 - alba/cli - 0 - info - Albamgr_client.make_client :sas-back-abm
2019-07-17 15:27:06 515326 +0800 - node1 - 22276/0000 - alba/cli - 1 - info - Albamgr_client.make_client :sas-back-abm
2019-07-17 15:27:06 516047 +0800 - node1 - 22276/0000 - alba/cli - 2 - info - Connecting to ADDR_INET(192.168.0.34,26408)
2019-07-17 15:27:06 516176 +0800 - node1 - 22276/0000 - alba/cli - 3 - info - Connecting to ADDR_INET(192.168.0.34,26408)
2019-07-17 15:27:06 516228 +0800 - node1 - 22276/0000 - alba/cli - 4 - info - connect_with 192.168.0.34 26408 None Net_fd.TCP (fd:8) succeeded
2019-07-17 15:27:06 516339 +0800 - node1 - 22276/0000 - alba/cli - 5 - info - connect_with 192.168.0.34 26408 None Net_fd.TCP (fd:3) succeeded

But this device is still in list-osds.

Best regards,
Yongsheng

[yongshengma]

There was this error all the time before I removed and reclaimed a hdd.
Jul 17 11:36:28 node1 alba: 2019-07-17 11:36:28 238863 +0800 - node1 - 34177/0000 - alba/maintenance - 4018 - info - connect_with failed: 192.168.20.33 8601 None Net_fd.TCP (fd:44): (Unix.Unix_error "Connection refused" connect "")
Now it's gone. I 'm not sure which operation took effect. I guess it's ok to see that device is still in list as there's decommissioned field. Thanks!

These logs still keep on showing up:

Jul 18 10:18:49 node1 alba[3606]: 2019-07-18 10:18:49 684183 +0800 - node1 - 3606/0000 - alba/proxy - 87431 - info - wKpwkiXci7jHuDfAfFwnLCploMijii1e "Asd_client.BadLongId(\"wKpwkiXci7jHuDfAfFwnLCploMijii1e\", \"J98Ei4HdWR6L7oaJqq9TfjuqLrrZL7p8\")" was unforeseen
Jul 18 10:18:49 node1 alba[3606]: 2019-07-18 10:18:49 684206 +0800 - node1 - 3606/0000 - alba/proxy - 87432 - info - wKpwkiXci7jHuDfAfFwnLCploMijii1e "Asd_client.BadLongId(\"wKpwkiXci7jHuDfAfFwnLCploMijii1e\", \"J98Ei4HdWR6L7oaJqq9TfjuqLrrZL7p8\")": should_invalidate:true should_retry:false
Jul 18 10:18:49 node1 alba[3606]: 2019-07-18 10:18:49 684624 +0800 - node1 - 3606/0000 - alba/proxy - 87433 - info - Could not yet requalify osd 0, trying again in 60.000000 seconds: Asd_client.BadLongId("wKpwkiXci7jHuDfAfFwnLCploMijii1e", "J98Ei4HdWR6L7oaJqq9TfjuqLrrZL7p8")