stats: Improved sequencing documentation for server-side stats events and added tests. #7885

RyanBlaney · 2024-12-02T03:06:36Z

Response to Issue #7824

RELEASE NOTES:

Documentation Clarification: Clarified the expected sequencing of stats events in the server-side stats.Handler.
New Test Cases: Added tests to verify the event sequencing for various RPC types, ensuring consistency and accuracy in server-side stats handling.

…s to verify the order of server-side events.

codecov · 2024-12-02T03:11:20Z

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 82.25%. Comparing base (4c07bca) to head (b16e1fb).
Report is 131 commits behind head on master.

Additional details and impacted files

@@            Coverage Diff             @@
##           master    #7885      +/-   ##
==========================================
+ Coverage   81.84%   82.25%   +0.40%     
==========================================
  Files         377      392      +15     
  Lines       38120    39140    +1020     
==========================================
+ Hits        31201    32194     +993     
- Misses       5603     5610       +7     
- Partials     1316     1336      +20

Files with missing lines	Coverage Δ
stats/stats.go	`68.42% <ø> (ø)`

... and 126 files with indirect coverage changes

🚀 New features to boost your workflow:

❄ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

arjan-bal · 2024-12-02T09:34:22Z

stats/stats_test.go

+func (s *testServer) UnaryCall(
+	ctx context.Context,
+	in *testpb.SimpleRequest,
+) (*testpb.SimpleResponse, error) {


grpc-go follows the Google style guide. Based on this section in the style guide:

The signature of a function or method declaration should remain on a single line to avoid indentation confusion.

Please revert the changes made to wrap function definationations and calls throughout the PR.

Got it! I'll disable my autoformatter going forward or reconfigure it to follow the Google style guide.

arjan-bal · 2024-12-02T09:38:15Z

stats/stats_test.go

@@ -786,8 +884,14 @@ func checkConnEnd(t *testing.T, d *gotData) {
 	st.IsClient() // TODO remove this.
 }

+type event struct {
+	eventType string
+	timestamp time.Time


What is the use of storing a timestamp? I believe we're only interested in the ordering of the events and not the absolute times they were emitted.

The timestamp was included for debugging purposes in case of test failures, since it could help identify delays or other issues. If you feel this isn’t valuable, I can to remove it for simplicity.

RyanBlaney · 2024-12-02T21:55:00Z

I have fixed the wrapping errors for formatting and I removed the timestamps from the event structs. Thank you for your feedback @arjan-bal, is there anything else you need me to do?

…. Added a server stats Server Stream RPC test.

stats/stats.go

stats/stats_test.go

purnesh42H · 2024-12-06T07:10:20Z

stats/stats_test.go

@@ -81,6 +83,55 @@ var (
 	}
 	// The id for which the service handler should return error.
 	errorID int32 = 32202
+
+    // Server Stats


Please fix the documentation everywhere. Remove extra spaces at start and end etc.

Everything should be resolved now. Thank you for your help!

purnesh42H · 2025-02-04T04:07:31Z

@RyanBlaney is it ready for re-review? I see that you have commits for the last comments.

Also, since its been called out, you can revert the style changes for want/expect in existing code but keep want/wantData in the new code that you have written. This is just to make code review focused on new changes. Thanks.

RyanBlaney · 2025-02-04T08:24:08Z

Yes, it should be ready! Everything new that I added is using want instead of expected.

dfawley · 2025-02-04T20:59:00Z

stats/stats_test.go

+
+		select {
+		case <-ctx.Done():
+			t.Fatalf("Timed out waiting for events to propagate: Diff (-got +want):\n%s", cmp.Diff(eventCount, len(wantUnarySequence)))


These are just single integers, so using cmp.Diff is just going to make it hard to read. Please change to:

Suggested change

t.Fatalf("Timed out waiting for events to propagate: Diff (-got +want):\n%s", cmp.Diff(eventCount, len(wantUnarySequence)))

t.Fatalf("Timed out waiting for events to propagate. Got %v events; want %v", eventCount, len(wantUnarySequence))

dfawley · 2025-02-04T20:59:20Z

stats/stats_test.go

+
+		select {
+		case <-ctx.Done():
+			t.Fatalf("Timed out waiting for events to propagate: Diff (-got +want):\n%s", cmp.Diff(eventCount, len(wantClientStreamSequence)))


dfawley · 2025-02-04T21:06:30Z

stats/stats_test.go

 	h.mu.Lock()
 	defer h.mu.Unlock()
 	h.gotConn = append(h.gotConn, &gotData{ctx, s.IsClient(), s})
 }

 func (h *statshandler) HandleRPC(ctx context.Context, s stats.RPCStats) {
+	switch s.(type) {
+	case *stats.Begin:
+		h.recordEvent("Begin")


Could you define constants for these strings please?

stats/stats_test.go

RyanBlaney · 2025-02-19T07:32:14Z

This PR should be complete, although there are a few more changes I want to make for a second PR.

Implement all TODO tests

There are several TODO statements left throughout the code, so I will implement the tests as needed.
Test OutTrailer
Remove IsClient call (2 instances)

Possible logic error

The docs state that the CompressedLength should be equal to the Length if compression is disabled.
The Length in this case is the uncompressed InPayload data

	// TODO: check values of WireLength and RecvTime.
	if st.Length > 0 && st.CompressedLength == 0 {
		t.Fatalf("st.WireLength = %v with non-empty data, want <non-zero>",
			st.CompressedLength)
	}

Fix deprecated tests

Diagnostics:
grpc.RPCCompressor is deprecated: use encoding.RegisterCompressor instead. Will be supported throughout 1.x. [default]

Diagnostics:
grpc.NewGZIPCompressor is deprecated: use package encoding/gzip. [default]

Diagnostics:
grpc.RPCDecompressor is deprecated: use encoding.RegisterCompressor instead. Will be supported throughout 1.x. [default]

Diagnostics:
grpc.NewGZIPDecompressor is deprecated: use package encoding/gzip. [default]

Diagnostics:
grpc.WithBlock is deprecated: this DialOption is not supported by NewClient. Will be supported throughout 1.x. [default]

Diagnostics:
grpc.Dial is deprecated: use NewClient instead.  Will be supported throughout 1.x. [default]

github-actions · 2025-02-25T22:45:41Z

This PR is labeled as requiring an update from the reporter, and no update has been received after 6 days. If no update is provided in the next 7 days, this issue will be automatically closed.

dfawley · 2025-02-27T21:07:01Z

stats/stats.go

+// InHeader contains stats about header reception.
+//
+// - Server-side: The first stats event after the RPC request is received.
+// - Client-side: Occurs after the `OutPayload` event (after the request payload is sent).


This isn't quite true. InHeader comes any time after OutHeader. This is especially true for bidi- or client- streaming RPCs, since OutPayload isn't guaranteed to come right after OutHeader, like it is for unary or server-streaming RPCs.

Maybe exclude the note about client-side here, but you can add a note to OutHeader for Client-side that says something like Only occurs after 'Begin', as headers are always the first thing sent on a stream.

dfawley · 2025-02-27T21:08:54Z

stats/stats_test.go

@@ -800,13 +803,52 @@ func (h *statshandler) TagRPC(ctx context.Context, info *stats.RPCTagInfo) conte
 	return context.WithValue(ctx, rpcCtxKey{}, info)
 }

+// recordEvent records an event in the statshandler along with a timestamp.
+func (h *statshandler) recordEvent(eventType string) {


Do we need any of this now? I don't think anyone reads this field since the new tests were removed.

Nope! Forgot to remove this. Thanks!

Improved sequencing Documentation for the stats events and added test…

44abfd0

…s to verify the order of server-side events.

RyanBlaney changed the title ~~Documentation, Test: Improved sequencing documentation for server-side stats events and added tests.~~ *: Improved sequencing documentation for server-side stats events and added tests. Dec 2, 2024

RyanBlaney marked this pull request as ready for review December 2, 2024 03:13

arjan-bal changed the title *: Improved sequencing documentation for server-side stats events and added tests. stats: Improved sequencing documentation for server-side stats events and added tests. Dec 2, 2024

arjan-bal self-requested a review December 2, 2024 09:25

arjan-bal self-assigned this Dec 2, 2024

arjan-bal added the Type: Documentation Documentation or examples label Dec 2, 2024

arjan-bal added this to the 1.69 Release milestone Dec 2, 2024

arjan-bal requested changes Dec 2, 2024

View reviewed changes

arjan-bal assigned RyanBlaney and unassigned arjan-bal Dec 2, 2024

arjan-bal mentioned this pull request Dec 2, 2024

stats: document server-side stats event ordering and add tests #7824

Open

RyanBlaney added 2 commits December 2, 2024 13:47

Fix formatting issues according to Google style guide

10f6726

Removed unnecessary time stamps from event tracing

92ec341

Cleared up difference between client side and server side stat events…

71adf5e

…. Added a server stats Server Stream RPC test.

arjan-bal assigned arjan-bal and unassigned RyanBlaney Dec 3, 2024

purnesh42H assigned purnesh42H and unassigned arjan-bal Dec 3, 2024

arjan-bal requested review from purnesh42H and arjan-bal and removed request for arjan-bal December 3, 2024 08:01

purnesh42H modified the milestones: 1.69 Release, 1.70 Release Dec 5, 2024

purnesh42H reviewed Dec 6, 2024

View reviewed changes

purnesh42H assigned RyanBlaney and unassigned purnesh42H Dec 6, 2024

purnesh42H added the Status: Requires Reporter Clarification label Dec 6, 2024

github-actions bot removed the stale label Feb 4, 2025

purnesh42H assigned purnesh42H and dfawley and unassigned RyanBlaney and purnesh42H Feb 4, 2025

dfawley removed the Status: Requires Reporter Clarification label Feb 4, 2025

dfawley reviewed Feb 4, 2025

View reviewed changes

dfawley assigned RyanBlaney and unassigned dfawley Feb 4, 2025

dfawley added the Status: Requires Reporter Clarification label Feb 11, 2025

RyanBlaney requested a review from dfawley February 14, 2025 05:11

Removed redundant tests.

34b4c26

arjan-bal modified the milestones: 1.71 Release, 1.72 Release Feb 19, 2025

RyanBlaney removed their assignment Feb 19, 2025

github-actions bot added stale and removed stale labels Feb 25, 2025

eshitachandwani assigned dfawley Feb 26, 2025

eshitachandwani removed the Status: Requires Reporter Clarification label Feb 26, 2025

dfawley reviewed Feb 27, 2025

View reviewed changes

dfawley assigned RyanBlaney and unassigned dfawley Feb 27, 2025

dfawley added the Status: Requires Reporter Clarification label Feb 27, 2025

RyanBlaney added 3 commits March 3, 2025 13:30

Removed unused recordStats functionality

c96054a

Fixed incorrect documentation

f27375e

removed unused field 'events'

b16e1fb

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

stats: Improved sequencing documentation for server-side stats events and added tests. #7885

stats: Improved sequencing documentation for server-side stats events and added tests. #7885

RyanBlaney commented Dec 2, 2024

codecov bot commented Dec 2, 2024 •

edited

Loading

arjan-bal Dec 2, 2024

RyanBlaney Dec 2, 2024

arjan-bal Dec 2, 2024

RyanBlaney Dec 2, 2024

RyanBlaney commented Dec 2, 2024

purnesh42H Dec 6, 2024

RyanBlaney Dec 11, 2024

purnesh42H commented Feb 4, 2025 •

edited

Loading

RyanBlaney commented Feb 4, 2025

dfawley Feb 4, 2025

dfawley Feb 4, 2025

dfawley Feb 4, 2025

RyanBlaney commented Feb 19, 2025

github-actions bot commented Feb 25, 2025

dfawley Feb 27, 2025

dfawley Feb 27, 2025

RyanBlaney Mar 3, 2025

	t.Fatalf("Timed out waiting for events to propagate: Diff (-got +want):\n%s", cmp.Diff(eventCount, len(wantUnarySequence)))
	t.Fatalf("Timed out waiting for events to propagate. Got %v events; want %v", eventCount, len(wantUnarySequence))

stats: Improved sequencing documentation for server-side stats events and added tests. #7885

Are you sure you want to change the base?

stats: Improved sequencing documentation for server-side stats events and added tests. #7885

Conversation

RyanBlaney commented Dec 2, 2024

Response to Issue #7824

codecov bot commented Dec 2, 2024 • edited Loading

Codecov Report

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

RyanBlaney commented Dec 2, 2024

Choose a reason for hiding this comment

Choose a reason for hiding this comment

purnesh42H commented Feb 4, 2025 • edited Loading

RyanBlaney commented Feb 4, 2025

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

RyanBlaney commented Feb 19, 2025

Implement all TODO tests

Possible logic error

Fix deprecated tests

github-actions bot commented Feb 25, 2025

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

codecov bot commented Dec 2, 2024 •

edited

Loading

purnesh42H commented Feb 4, 2025 •

edited

Loading