-
Notifications
You must be signed in to change notification settings - Fork 330
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
DP with NAS and meet a ‘Model contains a trainable layer that Opacus doesn't currently support’ #576
Comments
Hi @Drxdx could you share the code that you're trying to run? |
model = models.resnet18(num_classes=10) #darts_v2_model = DartsSpace.load_searched_model('darts-v2', pretrained=True, download=True) print("============",type(darts_v2_model))print(type(model)) For a simple example,I use resnet18 for DP training, because resent18 contains BN layer, so it is not possible for differential privacy. If we use the ModuleValidator.fix(model, strcit = False) function, it will change the BN layer to the GN layer, so we can use it.But I use the model with Darts, where the fix() function is useless. In this case, we have a function GradSampleModule.is_supported(m) that returns True for Conv and False for BN with Resnet18. But with Darts, both Conv and Bn return False. This problem has been bothering me for a long time. I hope you can help me |
Ok, this is actually interesting. I've investigated this a bit, and it seems like this problem could appear for any deserialized model. The problem is, when the model is loaded by
This the leads to I'm not exactly sure what about serialization process makes is create new class instances, and not sure how commonplace this is. However, I don't see a good reason why we should keep references as keys, not strings. Any ideas why switching to strings could backfire? |
The same problem in the First layer conv2d. Resnet18 is work, but the darts model is not work. |
📚 Documentation
i want to use dp with NAS, when i use the pre-trained DARTS model , ModuleValidator.fix() function does't work! anyone meet this problem?
The text was updated successfully, but these errors were encountered: