Skip to content

[improve][misc] Upgrade debezium version from 1.9.7.Final to 3.2.0.Final#27

Open
storm-08 wants to merge 2 commits into
branch-4.0from
debezium_upgrade
Open

[improve][misc] Upgrade debezium version from 1.9.7.Final to 3.2.0.Final#27
storm-08 wants to merge 2 commits into
branch-4.0from
debezium_upgrade

Conversation

@storm-08
Copy link
Copy Markdown
Collaborator

@storm-08 storm-08 commented Oct 22, 2025

Fixes apache#24888

Motivation

Debezium 3.2.0.Final includes significant improvements over 1.9.7.Final, such as:

  • Better performance and stability
  • Support for newer database versions
  • Bug fixes and security updates
    Upgrading will improve Pulsar IO Debezium connector reliability and compatibility with modern database deployments.

Modifications

  • Upgraded Debezium dependency from 1.9.7.Final to 3.2.0.Final in the relevant module(s)
  • Upgraded the PostreSQL and MySQL driver versions to be compatible with new debezium version
  • Updated the numMessages in test case (PulsarDebeziumOracleSourceTest#testDebeziumOracleDbSource) from 39 to 2 as the new debezium source connector does not process the internal messages (related to SYS.LOGMNRLT). Previously, the value was 39, which was composed of 37 internal messages and 2 messages ( 1- Create the customer table and 1- Insertion into the table).
  • Refactored database history to schema history
  • Upgraded debezium mongodb, mysql, postgres image to 2.5.0.Final
  • Added new timestamp fields which were introduced in debezium 2.5
  • Replaced mongo with mongosh, mongo was deprecated in debezium 2.6.1.Final
  • Modified mongodb sourcetester to use the new config values
  • Modified mssql sourcetester to use the new config values
  • Modified mysql sourcetester to use the new config values
  • Modified postgres sourcetester to use the new config values
  • Modified oracle sourcetester to use the new config values

Verifying this change

  • Make sure that the change passes the CI checks.

(Please pick either of the following options)

This change is a trivial rework / code cleanup without any test coverage.

(or)

This change is already covered by existing tests, such as (please describe tests).

(or)

This change added tests and can be verified as follows:

(example:)

  • Added integration tests for end-to-end deployment with large payloads (10MB)
  • Extended integration test for recovery after broker failure

Does this pull request potentially affect one of the following parts:

If the box was checked, please highlight the changes

  • Dependencies (add or upgrade a dependency)
  • The public API
  • The schema
  • The default values of configurations
  • The threading model
  • The binary protocol
  • The REST endpoints
  • The admin CLI options
  • The metrics
  • Anything that affects deployment

Documentation

  • doc
  • doc-required
  • doc-not-needed
  • doc-complete

Matching PR in forked repository

PR in forked repository:

mukesh-ctds and others added 2 commits October 22, 2025 16:55
(cherry picked from commit 4788375)
* LS-1356: upgraded debezium version and postgres, mysql drivers

* Updating the numMessages in test case (internal messages not processed by source)

(cherry picked from commit c3cf5e0)
@storm-08 storm-08 self-assigned this Oct 22, 2025
@storm-08 storm-08 changed the title [Enhancement][dependency] Upgrade debezium version from 1.9.7.Final to 3.2.0.Final [improve][misc] Upgrade debezium version from 1.9.7.Final to 3.2.0.Final Oct 22, 2025
@storm-08 storm-08 requested a review from Srinathc October 22, 2025 12:20
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Development

Successfully merging this pull request may close these issues.

3 participants