Label the darn axes, NO BAD IDEAS #12
Comments
|
Want to make: lists/ word clouds based on the traits which have the most positive, most neutral, and most negative weights in each of the first 3 dimensions -- this should lead to a D&D style alignment chart (3x3) which will hopefully show a clear pattern? Maybe also do with characters? Pseudocode:
Using this tutorial: https://towardsdatascience.com/how-to-make-word-clouds-in-python-that-dont-suck-86518cdcb61f Saving visualizations here as I make them so I can hopefully tell/ remember what they are in the future: https://docs.google.com/presentation/d/1_kc36iI6B2OmsZlbMxLB0xiT0ePaQQ2qh7NykKqsefI/edit?usp=sharing I'm using this function in the file nextstep.py to make very basic word clouds, giving the wordcloud python package the scores for each trait in each row of V as if they were "frequencies" even though they are not and letting the built in function generate_from_frequencies interpret that however it may. This seems to cause spyder to crash pretty often, so I made it lower quality and take fewer words: For the first 6 rows of V, that results in: For reference, the weights in the relevant sigma matrix are as follows, they get pretty small by at least e.g. 15 in: [4571.60069027, 3977.77079978, 3148.95421275, 2330.72490479, Moving on to a more D-D like chart... Call like this for e.g. the 3rd row of V with means removed: d1_2,d2_2,d3_2 = make_dd_wordcloud_dicts(V2,2,col2), then do simple_wordcloud(d3_2) (etc.). Produces this for the first 3 rows of V with means removed, dividing the traits into ordered 3rds... the first 89 with the most negative/ least positive, then the next 89 middle words, then the final 88 most positive words, those get put into 3 dicts by the above function; the way they are ordered in the chart is the opposite, the MOST POSITIVE at the top, etc. Maybe that isn't a good way to do that, because the spread isn't the same for every row of V, so I should come up with a new strategy for that? |
|
Other visualization ideas
|
|
Meeting with Dodds notes
|
|
Looking at how the values in the rows of V2 (V^T) are distributed (per above comment/ conversation), using the version of V2 from running SVD with the overall mean (49.65 ish) removed via e.g. First row of V Second row of V Third row of V Actually I think this might be easier to see as a scatter plot? First row of V Second row of V Third row of V Fourth row of V Fifth row of V Sixth row of V Seventh row of V Eighth row of V Ninth row of V Tenth row of V 11th row of V 17th row of V 22nd row of V 27th row of V 52nd row of V 77th row of V 102nd row of V 202nd row of V 236th row of V (last row) My Interpretation
|
|
Looking at trait magnitude in the rows of V (V2), with the overall mean removed --> 1c5699d Using this code to render the bar charts:
Interpretation: there do seem to be some patterns in the lower dimensions with "outlying" traits, e.g. a sort of leadership style component (captain<->first-mate seems to come up a lot) and a physical component (thick<->thin and tall<->short). And, in the lower dimensions, gender, sexuality, and procreation seem to come up a lot. To make a chart for a specific row of V, use |
|
Meeting with Dodds
|
|
Subtracting the mean of 50, rather than the overall mean.
|
|
Only the trait from the indicated side (positive left, negative right) with the largest magnitudes for the first 3 dimensions (theoretical mean removed)
|
|
To do:
|
|
Relative size of dimensions (sigma values, theoretical mean removed, first 20 dimensions) |
|
Notes from talking with Dodds Oct 12
|
|
What can columns of V^T and rows of U tell us (as opposed to rows of V^T and columns of U)? The columns of V^T tell us how a single trait contributes to each dimension of traits. We can look through the values of the columns to see which dimension that trait is most important to. The rows of U tell us how well a single character is described by each dimension. We can look through the values in the row to see which dimension best describes that character. Plotting the first 3 traits -- diligent<->lazy, competent<->incompetent, disorganized<->self-disciplined -- with the highest magnitudes from dimension 1 (row 1 of V^T) in the first 15 dimensions:
"Reversing" the last trait that is backwards from the other two:
We would not expect similar traits to track each other perfectly unless they had identical meanings. But seeing where they converge and diverge can perhaps help us pinpoint how some dimensions differ from each other. Finding the order of the traits: "hard<->soft" and "hard<->soft 2" track each other for the first 15 dimensions (good -- sanity check) It looks like as the dimensions progress they track each other more poorly, maybe indicating that the dimensions get less meaningful as they progress -- at some point they represent quirks of our specific dataset rather than underlying structure; replicating our exact initial data isn't meaningful or important. Where do they diverge? So they look like they diverge after the 7 or 8th dimension -- maybe evidence to focus on the first few? |
From trying to come up with what visuals I want in the paper, it has become clear I absolutely can't avoid labeling the axes anymore. I keep not doing it because I'm worried I'll do it wrong. So this is the No Bad Ideas version. If it's stupid I'm sure Dodds will let me know.
Basic idea: Which traits are most important to each "dimensions"? That will be those traits which have the most extreme weights in each ROW of V. Which characters best exemplify each "dimension"? That will be the characters which have the most extreme weights in each COLUMN of U. How much more important is the first "dimension" compared to the second? That is given by the relevant WEIGHT in Sigma.
The text was updated successfully, but these errors were encountered: