Add or Remove Sites from Multisite SSO

[CAJarrett]

I inherited a vast network with multiple domains and subdomains, most of which do not require a login. However, all of these are in the network, and therefore are in the Multisite SSO. Anytime someone logs in to one of the sites that does require it, the loading time can take up to 20 seconds. Obviously this is a problem.

How can I remove specific sites from the plugin, but keep them in my WP Network?