Skip to content

feat(metrics): keep read_bytes_count unchanged and add experimental_read_bytes_count and transition tracking#4720

Draft
kislaykishore wants to merge 11 commits into
masterfrom
read-split
Draft

feat(metrics): keep read_bytes_count unchanged and add experimental_read_bytes_count and transition tracking#4720
kislaykishore wants to merge 11 commits into
masterfrom
read-split

Conversation

@kislaykishore
Copy link
Copy Markdown
Collaborator

Description

Reverted the gcs/read_bytes_count metric to stay unchanged (zero attributes, original description). Added a new attributed metric gcs/experimental_read_bytes_count tracking read bytes categorized by read_type attributes (Parallel, Random, Sequential, Unknown).

Additionally, introduced sequential-to-random read access transition reasons tracking:

  • Added a new custom counter metric gcs/experimental_sequential_to_random_transitions_count mapped to transition reasons: backward_seek, forward_seek, and initial_offset_non_zero.
  • Instrumented the read type classifier (ReadTypeClassifier) to dynamically classify and atomically switch state (using thread-safe CompareAndSwap) on sequential to random transitions.
  • Integrated a new mock-based unit test validation suite inside read_type_classifier_test.go and updated high-level integration test assertions in both gcs_metrics_test.go and metrics_test.go.

Link to the issue in case of a bug fix.

N/A (At user's direct request)

Testing details

  1. Manual - Ran unified compile, code generation, import formatting, vet, and style lint pipeline (make build) successfully with 0 warnings/issues.
  2. Unit tests - Executed unit test sweeps under metrics package (go test -v ./metrics/...) and gcsx package (go test -v ./internal/gcsx/... -run TestReadTypeClassifier) completely successfully.
  3. Integration tests - Executed file system integration test suite (go test -v ./internal/fs/... -run "TestGCSMetrics|TestReadFile"), fully verifying metrics behaviors under file cache hit/miss cycles, sequential readers, buffered prefetch pipelines, and parallel MRD downloads.

Any backward incompatible change? If so, please explain.

N/A (Reverted gcs/read_bytes_count back to its original signature, making it fully backwards-compatible for all active telemetry consumers).

@kislaykishore kislaykishore added the execute-integration-tests Run only integration tests label May 22, 2026
@codecov
Copy link
Copy Markdown

codecov Bot commented May 22, 2026

Codecov Report

❌ Patch coverage is 84.33735% with 26 lines in your changes missing coverage. Please review.
✅ Project coverage is 83.75%. Comparing base (11ac2fc) to head (2b400bd).
⚠️ Report is 1 commits behind head on master.

Files with missing lines Patch % Lines
metrics/metrics_test_utils.go 0.00% 7 Missing ⚠️
internal/gcsx/random_reader.go 85.00% 5 Missing and 1 partial ⚠️
internal/gcsx/inactive_timeout_reader.go 66.66% 2 Missing and 3 partials ⚠️
internal/gcsx/client_readers/range_reader.go 90.69% 4 Missing ⚠️
internal/gcsx/read_type_classifier.go 90.00% 2 Missing and 2 partials ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##           master    #4720      +/-   ##
==========================================
+ Coverage   83.70%   83.75%   +0.05%     
==========================================
  Files         168      168              
  Lines       20740    21057     +317     
==========================================
+ Hits        17360    17636     +276     
- Misses       2734     2772      +38     
- Partials      646      649       +3     
Flag Coverage Δ
unittests 83.75% <84.33%> (+0.05%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

…and contiguous read transitions. Also fix panic bug in metrics verification utils.
…t to read/experimental_read_type_transitions_count. Regenerate metric mappings and update all classifiers, mock implementations, and integration test cases.
…lassifier constructor, allowing removal of defensive nil checks across classifier calls
…peClassifier test calls to prevent mock/test handle omissions
…ers from ComputeSeqPrefetchWindowAndAdjustType to reduce cyclomatic complexity
…a single transitionTo method, fully eliminating code duplication
@kislaykishore kislaykishore force-pushed the read-split branch 2 times, most recently from 4984809 to 4ac9fe3 Compare May 26, 2026 10:11
…andom preemption, explicit_close, forced_recreate, and inactive_timeout, add testify unit test suites, and write developer metrics documentation
…cancellations, define DRY config reasons YAML anchor, and add complete OTel/testify tests suites
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

execute-integration-tests Run only integration tests

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant