Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

VReplication: Optimize replication on target tablets #17166

Open
wants to merge 18 commits into
base: main
Choose a base branch
from
Open

VReplication: Optimize replication on target tablets #17166

wants to merge 18 commits into from

Commits on Nov 7, 2024

  1. Enable VPlayerBatching by default 

    Signed-off-by: Matt Lord <mattalord@gmail.com>
    @mattlord
    mattlord committed Nov 7, 2024
    Configuration menu
    Copy the full SHA
    23024bd View commit details
    Browse the repository at this point in the history

  2. Optimize row replication 

    Signed-off-by: Matt Lord <mattalord@gmail.com>
    @mattlord
    mattlord committed Nov 7, 2024
    Configuration menu
    Copy the full SHA
    a5472f4 View commit details
    Browse the repository at this point in the history

  3. Correct rowInfo cache updates with skipped cols 

    Signed-off-by: Matt Lord <mattalord@gmail.com>
    @mattlord
    mattlord committed Nov 7, 2024
    Configuration menu
    Copy the full SHA
    ab65f0c View commit details
    Browse the repository at this point in the history

Commits on Nov 9, 2024

  1. Correct cache mgmt for schema changes and binlog_image=noblob 

    Signed-off-by: Matt Lord <mattalord@gmail.com>
    @mattlord
    mattlord committed Nov 9, 2024
    Configuration menu
    Copy the full SHA
    4371f80 View commit details
    Browse the repository at this point in the history

  2. Adjust help text 

    Signed-off-by: Matt Lord <mattalord@gmail.com>
    @mattlord
    mattlord committed Nov 9, 2024
    Configuration menu
    Copy the full SHA
    ac27fd5 View commit details
    Browse the repository at this point in the history

  3. Enable pprof for testing 

    Signed-off-by: Matt Lord <mattalord@gmail.com>
    @mattlord
    mattlord committed Nov 9, 2024
    Configuration menu
    Copy the full SHA
    ad8233e View commit details
    Browse the repository at this point in the history

  4. Merge remote-tracking branch 'origin/main' into vrepl_target_perf 

    Signed-off-by: Matt Lord <mattalord@gmail.com>
    @mattlord
    mattlord committed Nov 9, 2024
    Configuration menu
    Copy the full SHA
    bf930af View commit details
    Browse the repository at this point in the history

Commits on Nov 10, 2024

  1. Get rid of the additional row/col info slice altogether 

    Signed-off-by: Matt Lord <mattalord@gmail.com>
    @mattlord
    mattlord committed Nov 10, 2024
    Configuration menu
    Copy the full SHA
    11b3493 View commit details
    Browse the repository at this point in the history

  2. Restore pprof_http default 

    Signed-off-by: Matt Lord <mattalord@gmail.com>
    @mattlord
    mattlord committed Nov 10, 2024
    Configuration menu
    Copy the full SHA
    ed81da1 View commit details
    Browse the repository at this point in the history

  3. colInfo type is no longer needed 

    Signed-off-by: Matt Lord <mattalord@gmail.com>
    @mattlord
    mattlord committed Nov 10, 2024
    Configuration menu
    Copy the full SHA
    fcfa376 View commit details
    Browse the repository at this point in the history

  4. VPlayerBatching is now enabled by default in all tests 

    Signed-off-by: Matt Lord <mattalord@gmail.com>
    @mattlord
    mattlord committed Nov 10, 2024
    Configuration menu
    Copy the full SHA
    2e3334f View commit details
    Browse the repository at this point in the history

Commits on Nov 11, 2024

  1. Add comment about field number gap in topodata.FieldEvent 

    Signed-off-by: Matt Lord <mattalord@gmail.com>
    @mattlord
    mattlord committed Nov 11, 2024
    Configuration menu
    Copy the full SHA
    aa359bc View commit details
    Browse the repository at this point in the history

  2. Add unit test 

    Signed-off-by: Matt Lord <mattalord@gmail.com>
    @mattlord
    mattlord committed Nov 11, 2024
    Configuration menu
    Copy the full SHA
    5f5c76a View commit details
    Browse the repository at this point in the history

  3. Deflake vdiffs in e2e tests using load generator 

    This impacted e2e tests that were not previously using VPlayerBatching.

The load generator constantly generates INSERTs, which are then
effeciently batched in vplayer so we get ~ 7x more throughput than
before and thus we need more time for filtered replication to catch
up after we've stopped it for the vdiff. ESPECIALLY since we're
using the --update-table-stats flag and the ANALYZE TABLE and its
locking causes a pause in updates to the table the load generator is
inserting into -- in particular for the test clusters that only
have PRIMARY tablets as everything is interacting directly.

Signed-off-by: Matt Lord <mattalord@gmail.com>
    @mattlord
    mattlord committed Nov 11, 2024
    Configuration menu
    Copy the full SHA
    f0f61db View commit details
    Browse the repository at this point in the history

  4. Revert Fields and bindLocations comparison change 

    It failed for some materializations

Signed-off-by: Matt Lord <mattalord@gmail.com>
    @mattlord
    mattlord committed Nov 11, 2024
    Configuration menu
    Copy the full SHA
    21564fd View commit details
    Browse the repository at this point in the history

  5. Adjust unit test 

    Signed-off-by: Matt Lord <mattalord@gmail.com>
    @mattlord
    mattlord committed Nov 11, 2024
    Configuration menu
    Copy the full SHA
    b8a3271 View commit details
    Browse the repository at this point in the history

  6. Merge remote-tracking branch 'origin/main' into vrepl_target_perf 

    Signed-off-by: Matt Lord <mattalord@gmail.com>
    @mattlord
    mattlord committed Nov 11, 2024
    Configuration menu
    Copy the full SHA
    5530030 View commit details
    Browse the repository at this point in the history

Commits on Nov 12, 2024

  1. Unit test tweaks 

    Signed-off-by: Matt Lord <mattalord@gmail.com>
    @mattlord
    mattlord committed Nov 12, 2024
    Configuration menu
    Copy the full SHA
    5f15e3f View commit details
    Browse the repository at this point in the history