Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

About Channel-Wise Attention(CWA) in the code. #5

Closed
Hoodythree opened this issue May 13, 2020 · 5 comments
Closed

About Channel-Wise Attention(CWA) in the code. #5

Hoodythree opened this issue May 13, 2020 · 5 comments
Labels
Good issue for better understanding

Comments

@Hoodythree
Copy link

Great work!But I have little question about CWA.In the origianl paper, I see M_i = diag(Mask_i), where diag is putting a vector on the princial diagonal of a diagonal matrix.But in the code below:

foo = [1] * 2 + [0] *  1
bar = []
for i in range(200):
    random.shuffle(foo)
    bar += foo
bar = [bar for i in range(nb_batch)]

I think bar is not a diagonal matrix. Please point out my problem if I misunderstood the operation here.Thanks a lot.

@dongliangchang
Copy link
Member

dot([0,1,0].t(), [1,2,3].t()) = [0,2,0].t()
matrix(diag([0,1,0].t()), [1,2,3].t()) = [0,2,0].t()
So, the code and the paper is consistent.
CWA could be visualized as a random channel-dropping operation in each channel group.

@dongliangchang
Copy link
Member

535949063158081301

@dongliangchang dongliangchang added the Good issue for better understanding label May 13, 2020
@dongliangchang dongliangchang pinned this issue May 13, 2020
@dongliangchang dongliangchang unpinned this issue May 13, 2020
@Hoodythree
Copy link
Author

Got it.Thanks for the response.

@dongliangchang dongliangchang pinned this issue Aug 6, 2020
@AND2797
Copy link

AND2797 commented Aug 6, 2020

foo = [1] * 2 + [0] *  1
bar = []
for i in range(200):
    random.shuffle(foo)
    bar += foo
bar = [bar for i in range(nb_batch)]

I had a small question about the code, here foo = [1] * 2 + [0] * 1 is only good for 3 channels correct? If we want to increase the number of channels (say 5), then foo = [1] * 3 + [0] * 2 is necessary, correct?

@dongliangchang
Copy link
Member

Yes. Please see Section III.A The Discriminality Component for details.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Good issue for better understanding
Projects
None yet
Development

No branches or pull requests

3 participants