Small things needs to be noted when using FaceIDplus

[Sooplex]

The output img of FaceAnalysis is BGR image,but the codes in IP-Adapter-FaceID and hugging face docs both use it as the input for the CLIP image encoder (it should be an RGB image).
It may lead to some promblems like unexpected blue hair.

[alexblattner]

@Sooplex you are correct. This works in general though:

ref_images_embeds = []
ip_adapter_images = []
innerip=cv2.cvtColor(np.array(image), cv2.COLOR_RGB2BGR)
faces=IF.get(innerip)

ip_adapter_images.append(face_align.norm_crop(innerip, landmark=faces[0].kps, image_size=224))
innerip = torch.from_numpy(faces[0].normed_embedding)
ref_images_embeds.append(innerip.unsqueeze(0))
ref_images_embeds = torch.stack(ref_images_embeds, dim=0).unsqueeze(0)
neg_ref_images_embeds = torch.zeros_like(ref_images_embeds)
id_embeds = torch.cat([neg_ref_images_embeds, ref_images_embeds]).to(dtype=torch.float16, device="cuda")

[blatr]

@alexblattner

I'm not sure that this snippet is not quite correct. If we take hugginface code then your code will save crops into ip_adapter_images list in BGR format and pass it to prepare_ip_adapter_image_embeds fn.

...
#insightface embeddings and crop extraction
...
clip_embeds = pipeline.prepare_ip_adapter_image_embeds(
  [ip_adapter_images], None, torch.device("cuda"), num_images, True)[0]

pipeline.unet.encoder_hid_proj.image_projection_layers[0].clip_embeds = clip_embeds.to(dtype=torch.float16)
pipeline.unet.encoder_hid_proj.image_projection_layers[0].shortcut = False # True if Plus v2

However if i'm not mistaken, prepare_ip_adapter_image_embeds() function accept RGB image.

I would rather suggest to get insightface embs from BGR image but crop original RGB one

image = np.array(image)
innerip_bgr= image[...,::-1] #to bgr
faces=IF.get(innerip_bgr)

ip_adapter_images.append(face_align.norm_crop(image, landmark=faces[0].kps, image_size=224))
...