Bug Report: Pointing vttablet to different (unmaged) MySQL makes it get stuck in loop about vitess

[mattlord]

OK, I'm unable to repeat what you saw (I modified the test case above to set sql_mode instead of gtid_mode and I did not see any problems). I'm not sure what part I may be missing. I'm going to close this for now but if you have more info we can re-open it at any time.

[mattlord]

@wiebeytec To clarify, you stopped the vttablet process that was using the old --db_host value and started a new vttablet process with the new --db_host value?

My guess would be that it's trying to perform an operation that it cannot. Can you please share the full list of flags that you're using, with sensitive info redacted as necessary? I'm guessing it's related to this: #14871

[wiebeytec]

First: I figured out what was wrong, and it was my mistake. So, the issue is one of logging errors. First to answer your questions:

@wiebeytec To clarify, you stopped the vttablet process that was using the old --db_host value and started a new vttablet process with the new --db_host value?

Indeed. I just stopped it and started it again, with a new --db-host.

My guess would be that it's trying to perform an operation that it cannot. Can you please share the full list of flags that you're using, with sensitive info redacted as necessary? I'm guessing it's related to this: #14871

I edited the top comment with the information.

It also seemed to me it was trying to do something it cannot, but it didn't log that.

The problem was that I forgot to pick the correct 'parameter group' for the settings. So it's one of these settings that made it fail:

[mattlord]

Thanks, @wiebeytec . So not being able to start makes sense, but not having any logs for it does not. We should be able to repeat it easily with the local examples by editing one of the tablet's my.cnf files to add gtid_mode=OFF_PERMISSIVE, then restarting it. We could then turn this into a bug report about the lack of any logging for that and address that specific part.

[mattlord]

@wiebeytec Here's a test case which should demonstrate the lack of a clear error message:

git checkout main && make build

pushd examples/local

./101_initial_cluster.sh

../common/scripts/vtorc-down.sh

vtctldclient SetKeyspaceDurabilityPolicy --durability-policy=none commerce

sleep 10

replicauid=$(vtctldclient GetTablets --keyspace commerce --tablet-type replica | awk '{print $1}' | cut -d- -f2 | bc)
primaryuid=$(vtctldclient GetTablets --keyspace commerce --tablet-type primary | awk '{print $1}' | cut -d- -f2 | bc)

CELL=zone1 KEYSPACE=commerce TABLET_UID="${replicauid}" ../common/scripts/vttablet-down.sh

# 102 is always the rdonly tablet
CELL=zone1 KEYSPACE=commerce TABLET_UID=102 ../common/scripts/vttablet-down.sh

echo "gtid_mode=OFF_PERMISSIVE" >> "${VTDATAROOT}/vt_$(vtctldclient GetTablets --keyspace commerce --tablet-type primary | awk '{print $1}' | cut -d '-' -f2)"/my.cnf

CELL=zone1 KEYSPACE=commerce TABLET_UID="${primaryuid}" ../common/scripts/vttablet-down.sh

CELL=zone1 KEYSPACE=commerce TABLET_UID="${primaryuid}" ../common/scripts/vttablet-up.sh

vtpid=$(pidof vttablet | awk '{print $1}')

tail -100f ${VTDATAROOT}/tmp/vttablet*.INFO.*.${vtpid}

But clearly there's something missing here.

What MySQL config option(s) in this parameter group did you think were the cause?

Are you using the Vitess Operator here?

[wiebeytec]

(We're not using Vitess Operator.)

I confirmed it was SQL_MODE value STRICT_TRANS_TABLES.