Branching for backward compatibilty: 8.7 and later about rally-tracks

[gbanasiak]

Same for 624d547.

[gbanasiak]

I've created 8.11 branch and raised #477 to stop further automatic updates in 8.7 branch.

[gbanasiak]

#478 addressed elastic/logs. I've tested this against 8.10.4 cluster as follows (based on this IT test sequence):

# sample data ingest
esrally race --track="elastic/logs" --challenge="logging-indexing" --enable-assertions --kill-running-processes --on-error="abort" --pipeline="benchmark-only" --target-hosts=${ES_HOST}:443 --test-mode --track-params="start_date:2021-01-01T00-00-00Z,end_date:2021-01-01T00-00-02Z,max_total_download_gb:18,raw_data_volume_per_day:72GB,max_generated_corpus_size:1GB,wait_for_status:green,force_data_generation:true,number_of_shards:2,number_of_replicas:0" --client-options="use_ssl:true,api_key:${ES_API_KEY}"

# verify indexing-querying
esrally race --track="elastic/logs" --challenge="logging-querying" --enable-assertions --exclude-tasks="tag:setup" --kill-running-processes --on-error="abort" --pipeline="benchmark-only" --target-hosts=${ES_HOST}:443 --test-mode --track-params="start_date:2021-01-01T00-00-00Z,end_date:2021-01-01T00-00-02Z,max_total_download_gb:18,raw_data_volume_per_day:72GB,max_generated_corpus_size:1GB,wait_for_status:green,force_data_generation:true,number_of_shards:2,number_of_replicas:0,query_warmup_time_period:1,query_time_period:1,workflow_time_interval:1,think_time_interval:1" --client-options="use_ssl:true,api_key:${ES_API_KEY}"

Rally logs and output confirm ESQL additions are not included:

2023-10-18 12:46:14,237 ActorAddr-(T|:63868)/PID:56503 esrally.racecontrol INFO Automatically derived distribution flavor [default], version [8.10.4], and build hash [b4a62ac808e886ff032700c391f45f1408b2538c]
2023-10-18 12:46:14,915 ActorAddr-(T|:63868)/PID:56503 esrally.utils.repo INFO Checking out [8.7] in [/Users/grzegorz/.rally/benchmarks/tracks/default] for distribution version [8.10.4].

When it comes to nyc_taxis, the #479 revert only removed the most recent update of esql challenge, but I think we should completely remove this challenge from 8.7 branch. WDYT @inqueue ?

[gbanasiak]

cc @craigtaverner

[gbanasiak]

@craigtaverner agreed on complete ESQL challenge removal in 8.7 branch so I've raised #483.