Skip to content

fix(nodes): add thresholding to lineart & lineart anime nodes #6869

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
Sep 17, 2024
Merged

fix(nodes): add thresholding to lineart & lineart anime nodes #6869

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
12 changes: 9 additions & 3 deletions invokeai/backend/image_util/lineart.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -214,8 +214,14 @@ def run(self, image: Image.Image) -> Image.Image:
line = line.cpu().numpy()
line = (line * 255.0).clip(0, 255).astype(np.uint8)

detected_map = line
detected_map = 255 - line

detected_map = 255 - detected_map
# The lineart model often outputs a lot of almost-black noise. SD1.5 ControlNets seem to be OK with this, but
# SDXL ControlNets are not - they need a cleaner map. 12 was experimentally determined to be a good threshold,
# eliminating all the noise while keeping the actual edges. Other approaches to thresholding may be better,
# for example stretching the contrast or removing noise.
detected_map[detected_map < 12] = 0

return np_to_pil(detected_map)
output = np_to_pil(detected_map)

return output
10 changes: 8 additions & 2 deletions invokeai/backend/image_util/lineart_anime.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -260,8 +260,14 @@ def run(self, image: Image.Image) -> Image.Image:
line = cv2.resize(line, (width, height), interpolation=cv2.INTER_CUBIC)
line = line.clip(0, 255).astype(np.uint8)

detected_map = line
detected_map = 255 - detected_map
detected_map = 255 - line

# The lineart model often outputs a lot of almost-black noise. SD1.5 ControlNets seem to be OK with this, but
# SDXL ControlNets are not - they need a cleaner map. 12 was experimentally determined to be a good threshold,
# eliminating all the noise while keeping the actual edges. Other approaches to thresholding may be better,
# for example stretching the contrast or removing noise.
detected_map[detected_map < 12] = 0

output = np_to_pil(detected_map)

return output
Loading