Skip to content

CAMEL-22539: Fix and re-enable 48 flaky tests across multiple modules#23022

Merged
gnodet merged 5 commits intoapache:mainfrom
gnodet:CAMEL-22539-fix-flaky-tests-batch2
May 7, 2026
Merged

CAMEL-22539: Fix and re-enable 48 flaky tests across multiple modules#23022
gnodet merged 5 commits intoapache:mainfrom
gnodet:CAMEL-22539-fix-flaky-tests-batch2

Conversation

@gnodet
Copy link
Copy Markdown
Contributor

@gnodet gnodet commented May 6, 2026

CAMEL-22539

Summary

Fix and re-enable 48 flaky tests across multiple modules that were disabled on CI via @DisabledIfSystemProperty(named = "ci.env.name"). This is a follow-up to PR #23020 which fixed 13 flaky tests in camel-core.

camel-management (4 tests)

  • AbstractManagedThrottlerTest: replace Thread.sleep(500) with Awaitility, increase timing tolerance (1200→2500ms), increase NotifyBuilder timeout (2→5s)
  • ManagedAConcurrentThrottlerTest: increase timing tolerance
  • ManagedAThrottlerTest: re-enable
  • ManagedAggregateControllerTest: add Awaitility wait for pending aggregation groups to prevent race condition

AWS Localstack infrastructure + 20 tests

  • AWSContainer: replace unreliable Wait.forLogMessage(".*Ready\\.\n", 1) with Wait.forHttp("/_localstack/health").forStatusCode(200) — the log-based strategy returns before services are actually ready
  • Re-enable all 20 disabled AWS integration tests (parameter-store, secrets-manager, kinesis, kms, lambda, sns)

camel-jms (7 tests)

  • JmsMultipleConsumersQueueTest/TopicTest: increase waitForConnections uptime check (100→500ms)
  • Re-enable all 7 disabled JMS tests

camel-nats (9 tests)

  • Add explicit setAssertPeriod(5000) before mock assertions to accommodate async NATS message delivery
  • Re-enable all 9 disabled NATS integration tests

Remaining components (6 tests)

  • GrpcProducerStreamingTest: replace Thread.sleep(2000) with Awaitility
  • KafkaConsumerAuthInvalidIT: increase assertIsSatisfied timeout (3→10s)
  • Re-enable: HttpNoConnectionTest, MllpTcpClientProducerConnectionErrorTest, SjmsToDTest, UndertowWsConsumerRouteTest

Test plan

  • All non-container tests compile and pass locally
  • Formatter applied to all affected modules
  • CI passes with all tests re-enabled

camel-management (4 tests):
- AbstractManagedThrottlerTest: replace Thread.sleep with Awaitility, increase timing tolerance, increase NotifyBuilder timeout
- ManagedAConcurrentThrottlerTest: increase timing tolerance
- ManagedAThrottlerTest: remove disable annotation
- ManagedAggregateControllerTest: add Awaitility wait for pending aggregation groups

AWS Localstack (20 tests):
- AWSContainer: replace unreliable log-based wait strategy with HTTP health check on /_localstack/health
- Re-enable all 20 disabled AWS integration tests (parameter-store, secrets-manager, kinesis, kms, lambda, sns)

camel-jms (7 tests):
- JmsMultipleConsumersQueueTest/TopicTest: increase waitForConnections uptime check
- Re-enable all 7 disabled JMS tests

camel-nats (9 tests):
- Add explicit setAssertPeriod(5000) before mock assertions for async delivery
- Re-enable all 9 disabled NATS tests

Remaining components (6 tests):
- GrpcProducerStreamingTest: replace Thread.sleep with Awaitility
- KafkaConsumerAuthInvalidIT: increase assertion timeout
- Re-enable HttpNoConnectionTest, MllpTcpClientProducerConnectionErrorTest, SjmsToDTest, UndertowWsConsumerRouteTest

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@github-actions
Copy link
Copy Markdown
Contributor

github-actions Bot commented May 6, 2026

🌟 Thank you for your contribution to the Apache Camel project! 🌟
🤖 CI automation will test this PR automatically.

🐫 Apache Camel Committers, please review the following items:

  • First-time contributors require MANUAL approval for the GitHub Actions to run
  • You can use the command /component-test (camel-)component-name1 (camel-)component-name2.. to request a test from the test bot although they are normally detected and executed by CI.
  • You can label PRs using skip-tests and test-dependents to fine-tune the checks executed by this PR.
  • Build and test logs are available in the summary page. Only Apache Camel committers have access to the summary.

⚠️ Be careful when sharing logs. Review their contents before sharing them publicly.

gnodet and others added 2 commits May 6, 2026 23:56
The Awaitility block introduced a scoped variable that shadowed the
original 'pending' variable, leaving later references unresolved.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@github-actions
Copy link
Copy Markdown
Contributor

github-actions Bot commented May 7, 2026

🧪 CI tested the following changed modules:

  • components/camel-aws/camel-aws-parameter-store
  • components/camel-aws/camel-aws-secrets-manager
  • components/camel-aws/camel-aws2-kinesis
  • components/camel-aws/camel-aws2-kms
  • components/camel-aws/camel-aws2-lambda
  • components/camel-aws/camel-aws2-sns
  • components/camel-grpc
  • components/camel-http
  • components/camel-jms
  • components/camel-kafka
  • components/camel-mllp
  • components/camel-nats
  • components/camel-sjms
  • components/camel-undertow
  • core/camel-management
  • test-infra/camel-test-infra-aws-v2

ℹ️ Dependent modules were not tested because the total number of affected modules exceeded the threshold (50). Use the test-dependents label to force testing all dependents.

⚠️ Some tests are disabled on GitHub Actions (@DisabledIfSystemProperty(named = "ci.env.name")) and require manual verification:

  • components/camel-jms: 1 test(s) disabled on GitHub Actions
  • components/camel-kafka: 2 test(s) disabled on GitHub Actions
  • components/camel-mllp: 1 test(s) disabled on GitHub Actions
Build reactor — dependencies compiled but only changed modules were tested (16 modules)
  • Camel :: AWS2 KMS
  • Camel :: AWS2 Kinesis
  • Camel :: AWS2 Lambda
  • Camel :: AWS2 Parameter Store
  • Camel :: AWS2 SNS
  • Camel :: AWS2 Secrets Manager
  • Camel :: HTTP
  • Camel :: JMS
  • Camel :: Kafka
  • Camel :: MLLP
  • Camel :: Management
  • Camel :: Nats
  • Camel :: Simple JMS
  • Camel :: Test Infra :: AWS SDK v2
  • Camel :: Undertow
  • Camel :: gRPC

⚙️ View full build and test results

Copy link
Copy Markdown
Contributor

@apupier apupier left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How many times have you launched each tests to increase the chance to detect if they are still flaky?

@gnodet gnodet marked this pull request as ready for review May 7, 2026 13:25
@gnodet gnodet merged commit 685e1a4 into apache:main May 7, 2026
6 checks passed
@gnodet
Copy link
Copy Markdown
Contributor Author

gnodet commented May 7, 2026

Claude Code on behalf of Guillaume Nodet

Each test was run 100 consecutive times locally (in a loop with fail-fast on first failure), all passing 100/100:

Module Tests Result
camel-management ManagedAggregateControllerTest, AbstractManagedThrottlerTest, ManagedAThrottlerTest, ManagedAConcurrentThrottlerTest 100/100
camel-mllp MllpTcpClientProducerConnectionErrorTest 100/100
camel-grpc GrpcProducerStreamingTest 100/100
camel-http HttpNoConnectionTest 100/100
camel-undertow UndertowWsConsumerRouteTest 100/100
camel-jms JmsComponentIbmMQTest, JmsMultipleConsumersQueueTest, JmsMultipleConsumersTopicTest, ManagedJmsEndpointTest, JmsReplyToIbmMQTest, TemporaryQueueRouteTest 100/100
camel-sjms SjmsToDTest 100/100

The same results were also posted on the JIRA issue.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants