Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Documentation on sub-models #95

Open
dan-tripp-siteimprove opened this issue Dec 2, 2022 · 9 comments
Open

Documentation on sub-models #95

dan-tripp-siteimprove opened this issue Dec 2, 2022 · 9 comments

Comments

@dan-tripp-siteimprove
Copy link

There's something that I don't understand in the documentation in pict.md, where it says:

"Placing all hardware parameters into one sub-model produces fewer distinct hardware configurations and potentially lowers the cost of testing."

But the example shown (with { PLATFORM, CPUS, RAM, HDD } @ 3) seems to produce more distinct hardware configurations. Not fewer.

By my count:
{ PLATFORM, CPUS, RAM, HDD } @ 3 produces 28 hardware configurations (across 336 generated tests.)
{ PLATFORM, CPUS, RAM, HDD } @ 2 produces 10 hardware configurations (across 120 generated tests.)
No sub-models (for neither hardware nor software) produces 17 hardware configurations (across 17 generated tests.)

So what's happening here? Am I looking at the PICT results wrong, or at the documentation wrong? Or could the documentation be improved by changing this:
{ PLATFORM, CPUS, RAM, HDD } @ 3
... to this:
{ PLATFORM, CPUS, RAM, HDD } @ 2

@jaccz
Copy link
Member

jaccz commented Dec 2, 2022 via email

@dan-tripp-siteimprove
Copy link
Author

Thank you for your response. Regarding the part "The trend is going to be more visible with larger models. E.g. add a few more parameters to it to see the real difference." - I tried that today and I'm afraid I don't follow. I've included an example model below, and some commented-out sub-models which I also tried. None of the sub-model variations produced fewer hardware configurations than my model without the sub-models. (No sub-models produced 20 hardware configurations, with var7 to var13). If, by chance, you could show me a model which shows the real difference, please do. I'm interested because as it stands, it seems that I don't understand sub-models. Maybe I'm not the only one.

PLATFORM: x86, x64, arm
CPUS: 1, 2, 4
RAM: 1GB, 4GB, 64GB
HDD: SCSI, IDE
separator: zzz
OS: Win7, Win8, Win10
Browser: Edge, Opera, Chrome, Firefox
APP: Word, Excel, Powerpoint
var7: x, y, z
var8: x, y, z
var9: x, y, z
var10: x, y, z
var11: x, y, z
var12: x, y, z
var13: x, y, z

{ PLATFORM, CPUS, RAM, HDD } @ 3
{ OS, Browser } @ 2

#{ PLATFORM, CPUS, RAM, HDD , var7, var8, var9, var10, var11, var12, var13 } @ 3
#{ OS, Browser , var7, var8, var9, var10, var11, var12, var13 } @ 2

@jaccz
Copy link
Member

jaccz commented Dec 5, 2022 via email

@dan-tripp-siteimprove
Copy link
Author

That cleared it up. Thank you. If I made a pull request that adds some of this discussion to pict.md, would you consider it?

@jaccz
Copy link
Member

jaccz commented Dec 5, 2022 via email

@dan-tripp-siteimprove
Copy link
Author

Right on. Now there's something else I'd like to ask, so that I can get the details of this PR correct:
You wrote "188 combinations of values to cover, 40 generated tests, 11 combinations of the first four parameters." But I see 10, not 11.

These are the 10 that I see:
arm 1 64GB IDE
arm 2 1GB SCSI
arm 2 4GB IDE
arm 4 64GB IDE
x64 1 4GB SCSI
x64 2 64GB IDE
x64 4 1GB SCSI
x86 1 1GB IDE
x86 2 64GB SCSI
x86 4 4GB IDE

To my surprise, if I use this model below (with no sub-model and no non-hardware parameters) then I see 11:
PLATFORM: x86, x64, arm
CPUS: 1, 2, 4
RAM: 1GB, 4GB, 64GB
HDD: SCSI, IDE

What do you think?

@jaccz
Copy link
Member

jaccz commented Dec 6, 2022 via email

@dan-tripp-siteimprove
Copy link
Author

Okay, I made a PR. I hope it helps. I should get permission from my employer before I click "agree" on the Contributor License Agreement. I'll try to do that later this week.

@dan-tripp
Copy link
Contributor

Closed by #112, I think.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants