Do you know if I am able to differentiate between the GPU types in rocminfo? I dont have amd so I can't tell. Ideally there should be a rocminfo flag to skip igpus

Yeah, that would be ideal, wouldn't it? :P rocminfo has no options, not even --help or --version. rocm-smi is a lot more detailed, but despite all the options it has, there is nothing I could see that screams "iGPU". You just sort of have to know that gfx1036 (Raphael) is an iGPU arch, for example. There is a CUDA API for getting device info that has an "integrated" flag, and even that just returns "0" for the iGPU as though it was discrete in the HIP emulation.

FWIW this is the output of rocm-smi --showproductname --showmeminfo vram --showuniqueid --csv:

device,Unique ID,VRAM Total Memory (B),VRAM Total Used Memory (B),Card Series,Card Model,Card Vendor,Card SKU,Subsystem ID,Device Rev,Node ID,GUID,GFX Version
card0,0x9246____________,17163091968,692142080,Navi 21 [Radeon RX 6800/6800 XT / 6900 XT],0x73bf,Advanced Micro Devices Inc. [AMD/ATI],001,0x2406,0xc1,1,45534,gfx1030
card1,N/A,67108864,26079232,Raphael,0x164e,Advanced Micro Devices Inc. [AMD/ATI],RAPHAEL,0x364e,0xc6,2,52156,gfx1036

I stubbed out the id of my discrete card just in case anyone could get up to shenanigans with it (I don't think this is actually the serial number because there's an option for that too, and that never returns anything, but who knows). So maybe the unique ID is usable as the iGPU has "N/A"? But I couldn't tell you if this is a guarantee.

Incidentally there is also a --json option, which is probably more reliable/convenient than hard-coding the CSV column order as is done now, if you're not actually parsing the CSV header. Here's how that looks (except the real output is unformatted):

{
  "card0": {
    "Unique ID": "0x9246____________",
    "VRAM Total Memory (B)": "17163091968",
    "VRAM Total Used Memory (B)": "708767744",
    "Card Series": "Navi 21 [Radeon RX 6800/6800 XT / 6900 XT]",
    "Card Model": "0x73bf",
    "Card Vendor": "Advanced Micro Devices, Inc. [AMD/ATI]",
    "Card SKU": "001",
    "Subsystem ID": "0x2406",
    "Device Rev": "0xc1",
    "Node ID": "1",
    "GUID": "45534",
    "GFX Version": "gfx1030"
  },
  "card1": {
    "Unique ID": "N/A",
    "VRAM Total Memory (B)": "67108864",
    "VRAM Total Used Memory (B)": "26079232",
    "Card Series": "Raphael",
    "Card Model": "0x164e",
    "Card Vendor": "Advanced Micro Devices, Inc. [AMD/ATI]",
    "Card SKU": "RAPHAEL",
    "Subsystem ID": "0x364e",
    "Device Rev": "0xc6",
    "Node ID": "2",
    "GUID": "52156",
    "GFX Version": "gfx1036"
  }
}

To be clear, I'm not suggesting koboldcpp should actually do something like detect iGPUs and actively ignore them -- this would be counterproductive for people who actually want to use them, which is absolutely possible for most AMD configurations (though you may need HSA_OVERRIDE_GFX_VERSION). AMD's iGPUs tend to be actually useful in practice for compute work, unlike Intel's (pre-Xe, at least). CUDA_VISIBLE_DEVICES/HIP_VISIBLE_DEVICES is pretty well established as a GPU selection tool and already restricts koboldcpp just fine too, it's just that the HW reporting tools ignore it (at least the AMD ones do, no clue about NV).

Decided to be un-lazy and cook up a PR anyway. The logic may or may not apply to NV as well, per my earlier comment.

Note that the calculation as a whole is, I think, wrong/suspect/deficient when multiple GPUs are used simultaneously, since there seems to be no attempt at using tensor_split, meaning the layer calc will default to the smallest GPU even if the model is being distributed. But that's another issue altogether.