Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Cache storage policy application results (Get/Head/GetRange) #2896

Merged
merged 3 commits into from
Aug 7, 2024

Conversation

cthulhu-rider
Copy link
Contributor

@cthulhu-rider cthulhu-rider commented Jul 17, 2024


i created 2 containers with 5 object in each and ran 30 routines sending 1000 GET queries (each time random object)

v0.42.1

Screenshot from 2024-07-18 16-27-59

current branch

Screenshot from 2024-07-18 16-35-55


Go script
strObjs := []string{
		"47pRu4GKKHL3yJ5wkXRDZj5QsyfH3wUUy27x9eEp7wgK/7GRiZmkxHruwrR5YbwfiTCMzo5rnEHr8HmyNAPwAEzex",
		"47pRu4GKKHL3yJ5wkXRDZj5QsyfH3wUUy27x9eEp7wgK/3ZSUCYqacwGY3teqJm3f9gDYSsEtB4uccXUp5in6jXBc",
		"47pRu4GKKHL3yJ5wkXRDZj5QsyfH3wUUy27x9eEp7wgK/EMdsnvaUK2XQtxLf1zFjVeFFtbszEwKu8mvYt9W15oMw",
		"47pRu4GKKHL3yJ5wkXRDZj5QsyfH3wUUy27x9eEp7wgK/En1K6HVfzoQtX9Vn2nW85szWduXqepdbmeAt9t2NMQJL",
		"47pRu4GKKHL3yJ5wkXRDZj5QsyfH3wUUy27x9eEp7wgK/B7UjiPD8UcCQsWHpDLZWswATnA7q4bWvbHAvJJXCtbPq",
		"2GN486btkTpxsYHsVjz5tZzug65YiSFUAYqAQiEkg4sg/FtRXxUBiTteyCM5QyeKKwFsJiJsNQGDMG31dTchagep3",
		"2GN486btkTpxsYHsVjz5tZzug65YiSFUAYqAQiEkg4sg/FJijkw5XauTY9N2TKUSTf8QxJoDjBFGVS9Qc9V6ejW1i",
		"2GN486btkTpxsYHsVjz5tZzug65YiSFUAYqAQiEkg4sg/HhbtK3cTcwu8uzLaMjMFg9Cnj1ZkmPAvPGPAXBNoYM6E",
		"2GN486btkTpxsYHsVjz5tZzug65YiSFUAYqAQiEkg4sg/DcLuSr69JEEgeqTE3AaMAQqcVzPDHxtK88ZXdDVycRfM",
		"2GN486btkTpxsYHsVjz5tZzug65YiSFUAYqAQiEkg4sg/Afykr7S6kgwbkVA5fsmw6bjkv8LrUQ7hxzLWreneoERh",
	}
	const endpoint = "s01.neofs.devenv:8080"
	const workers = 30
	ctx := context.Background()
	signer := test.RandomSignerRFC6979(t)

	objs := make([]oid.Address, len(strObjs))
	for i := range strObjs {
		require.NoError(t, objs[i].DecodeString(strObjs[i]))
	}

	c, err := client.New(client.PrmInit{})
	require.NoError(t, err)
	var prm client.PrmDial
	prm.SetServerURI(endpoint)
	require.NoError(t, c.Dial(prm))
	t.Cleanup(func() { c.Close() })

	var wg sync.WaitGroup
	st := time.Now()
	for i := 0; i < workers; i++ {
		wg.Add(1)
		go func() {
			defer wg.Done()

			for i := 0; i < 1000; i++ {
				if i%100 == 0 {
					fmt.Println(i)
				}
				a := objs[rand.Int()%len(objs)]
				_, rdr, err := c.ObjectGetInit(ctx, a.Container(), a.Object(), signer, client.PrmObjectGet{})
				if err != nil {
					log.Println("init search:", err)
					continue
				}
				rdr.Close()
			}
		}()
	}
	wg.Wait()
	fmt.Println(time.Since(st))

@cthulhu-rider cthulhu-rider force-pushed the optimize/get-policy branch 3 times, most recently from 19faa8a to 383e341 Compare July 18, 2024 11:52
@cthulhu-rider cthulhu-rider changed the title Cache storage policy application results (Get) Cache storage policy application results (Get/Head/GetRange) Jul 18, 2024
Copy link

codecov bot commented Jul 18, 2024

Codecov Report

Attention: Patch coverage is 72.72727% with 27 lines in your changes missing coverage. Please review.

Project coverage is 23.78%. Comparing base (8448ff3) to head (8b82bbd).

Files Patch % Lines
pkg/services/object/get/container.go 56.25% 11 Missing and 3 partials ⚠️
cmd/neofs-node/object.go 0.00% 5 Missing ⚠️
cmd/neofs-node/policy.go 91.52% 3 Missing and 2 partials ⚠️
pkg/services/object/get/service.go 0.00% 3 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##           master    #2896      +/-   ##
==========================================
+ Coverage   23.73%   23.78%   +0.05%     
==========================================
  Files         775      775              
  Lines       44933    44970      +37     
==========================================
+ Hits        10664    10698      +34     
  Misses      33417    33417              
- Partials      852      855       +3     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@cthulhu-rider cthulhu-rider force-pushed the optimize/get-policy branch 2 times, most recently from d24fb33 to 380a6b4 Compare July 18, 2024 13:16
@cthulhu-rider cthulhu-rider marked this pull request as ready for review July 18, 2024 13:19
cmd/neofs-node/policy.go Outdated Show resolved Hide resolved
cmd/neofs-node/policy.go Outdated Show resolved Hide resolved
var j, jLim uint
primary := true

for i := 0; i < len(nodeLists); i++ { // do not use for-range!
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why? the comment does not explain, so it is useless (or even harmful)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

usually range is used for this, but here it wouldn't work

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

only after reading it twice (to the end), i saw i = -1, why not mention it? it is a comment, it should help. any code can be changed, so comments should explain something, and if they are outdated, it should be easy to understand. not saying it is required here but it is a general rule i would say

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Tricky one. Can be simplified by inverting i/j loops: walk over the top ones from each list first and then go deeper till there are no more nodes to check. It can be more optimal in some cases (top nodes from different lists are still more likely to have the object), but maybe not for all of them (policies may have local nodes first).

The other way of course is just for _, primary := range []bool{true, false} and a regular range loop over nodeLists then. Less tricks, same meaning.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

top nodes from different lists are still more likely to have the object

they are homogeneous in terms of policies

for _, primary := range []bool{true, false}

i can make any style you think is readable. I'm fine with current. It can be simply divided into two loops. No tricks at all

pkg/services/object/get/service.go Outdated Show resolved Hide resolved
@@ -36,18 +48,24 @@ type containerNodes struct {
containers container.Source
network netmap.Source

cache *lru.Cache[containerNodesCacheKey, storagePolicyRes]
cache *lru.Cache[containerNodesCacheKey, storagePolicyRes]
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

rename too?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

no need, it contains container nodes

@carpawell
Copy link
Member

Some conflicts.

Continues 2f29338 for `ObjectService`'s
`Get`/`Head`/`GetRange` server handlers.

Signed-off-by: Leonard Lyubich <[email protected]>
Continues 10d05a4 for sorting container
nodes for objects.

Since each container may include plenty of objects, cache size limit is
chosen - again heuristically - 10 times bigger, i.e. 10K.

Refs #2692.

Signed-off-by: Leonard Lyubich <[email protected]>
They are vendored by NeoFS SDK, but the reaction to their unexpected
behavior also needs to be tested.

Signed-off-by: Leonard Lyubich <[email protected]>
@carpawell
Copy link
Member

What's the status here? Some threads are not fully answered and no review request yet.

@cthulhu-rider
Copy link
Contributor Author

What's the status here?

i've already processed all of them, pls ref me to the exact ones

@roman-khimov roman-khimov merged commit 504daae into master Aug 7, 2024
20 of 22 checks passed
@roman-khimov roman-khimov deleted the optimize/get-policy branch August 7, 2024 16:04
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants