v8.5.5 pd,tidb: support affinity schedule (#22270) (#22315)

Author: ti-chi-bot

TOC.md

@@ -881,6 +881,7 @@
       - [`SET ROLE`](/sql-statements/sql-statement-set-role.md)
       - [`SET TRANSACTION`](/sql-statements/sql-statement-set-transaction.md)
       - [`SET <variable>`](/sql-statements/sql-statement-set-variable.md)
+      - [`SHOW AFFINITY`](/sql-statements/sql-statement-show-affinity.md)
       - [`SHOW ANALYZE STATUS`](/sql-statements/sql-statement-show-analyze-status.md)
       - [`SHOW [BACKUPS|RESTORES]`](/sql-statements/sql-statement-show-backups.md)
       - [`SHOW BINDINGS`](/sql-statements/sql-statement-show-bindings.md)
@@ -998,6 +999,7 @@
     - [Temporary Tables](/temporary-tables.md)
     - [Cached Tables](/cached-tables.md)
     - [FOREIGN KEY Constraints](/foreign-key.md)
+    - [Table-Level Data Affinity](/table-affinity.md)
     - Character Set and Collation
       - [Overview](/character-set-and-collation.md)
       - [GBK](/character-set-gbk.md)

information-schema/information-schema-partitions.md

@@ -45,8 +45,9 @@ The output is as follows:
 | TABLESPACE_NAME               | varchar(64)  | YES  |      | NULL    |       |
 | TIDB_PARTITION_ID             | bigint(21)   | YES  |      | NULL    |       |
 | TIDB_PLACEMENT_POLICY_NAME    | varchar(64)  | YES  |      | NULL    |       |
+| TIDB_AFFINITY                 | varchar(128) | YES  |      | NULL    |       |
 +-------------------------------+--------------+------+------+---------+-------+
-27 rows in set (0.00 sec)
+28 rows in set (0.00 sec)
 ```
 
 ```sql
@@ -85,6 +86,7 @@ SUBPARTITION_ORDINAL_POSITION: NULL
               TABLESPACE_NAME: NULL
             TIDB_PARTITION_ID: 89
    TIDB_PLACEMENT_POLICY_NAME: NULL
+                TIDB_AFFINITY: NULL
 *************************** 2. row ***************************
                 TABLE_CATALOG: def
                  TABLE_SCHEMA: test
@@ -113,6 +115,7 @@ SUBPARTITION_ORDINAL_POSITION: NULL
               TABLESPACE_NAME: NULL
             TIDB_PARTITION_ID: 90
    TIDB_PLACEMENT_POLICY_NAME: NULL
+                TIDB_AFFINITY: NULL
 2 rows in set (0.00 sec)
 ```
 

information-schema/information-schema-tables.md

@@ -41,8 +41,12 @@ DESC tables;
 | TABLE_COMMENT             | varchar(2048) | YES  |      | NULL     |       |
 | TIDB_TABLE_ID             | bigint(21)    | YES  |      | NULL     |       |
 | TIDB_ROW_ID_SHARDING_INFO | varchar(255)  | YES  |      | NULL     |       |
+| TIDB_PK_TYPE              | varchar(64)   | YES  |      | NULL     |       |
+| TIDB_PLACEMENT_POLICY_NAME | varchar(64)  | YES  |      | NULL     |       |
+| TIDB_TABLE_MODE           | varchar(16)   | YES  |      | NULL     |       |
+| TIDB_AFFINITY             | varchar(128)  | YES  |      | NULL     |       |
 +---------------------------+---------------+------+------+----------+-------+
-23 rows in set (0.00 sec)
+27 rows in set (0.00 sec)
 ```
 
 {{< copyable "sql" >}}
@@ -72,10 +76,14 @@ SELECT * FROM tables WHERE table_schema='mysql' AND table_name='user'\G
                CHECK_TIME: NULL
           TABLE_COLLATION: utf8mb4_bin
                  CHECKSUM: NULL
-           CREATE_OPTIONS: 
-            TABLE_COMMENT: 
+           CREATE_OPTIONS:
+            TABLE_COMMENT:
             TIDB_TABLE_ID: 5
 TIDB_ROW_ID_SHARDING_INFO: NULL
+             TIDB_PK_TYPE: CLUSTERED
+TIDB_PLACEMENT_POLICY_NAME: NULL
+           TIDB_TABLE_MODE: Normal
+             TIDB_AFFINITY: NULL
 1 row in set (0.00 sec)
 ```
 
@@ -115,12 +123,16 @@ The description of columns in the `TABLES` table is as follows:
 * `CREATE_OPTIONS`: Creates options.
 * `TABLE_COMMENT`: The comments and notes of the table.
 
-Most of the information in the table is the same as MySQL. Only two columns are newly defined by TiDB:
+Most of the information in the table is the same as MySQL. The following columns are newly defined by TiDB:
 
 * `TIDB_TABLE_ID`: to indicate the internal ID of a table. This ID is unique in a TiDB cluster.
 * `TIDB_ROW_ID_SHARDING_INFO`: to indicate the sharding type of a table. The possible values are as follows:
     - `"NOT_SHARDED"`: the table is not sharded.
     - `"NOT_SHARDED(PK_IS_HANDLE)"`: the table that defines an integer Primary Key as its row id is not sharded.
     - `"PK_AUTO_RANDOM_BITS={bit_number}"`: the table that defines an integer Primary Key as its row id is sharded because the Primary Key is assigned with `AUTO_RANDOM` attribute.
     - `"SHARD_BITS={bit_number}"`: the table is sharded using `SHARD_ROW_ID_BITS={bit_number}`.
-    - NULL: the table is a system table or view, and thus cannot be sharded.
+    - `NULL`: the table is a system table or view, and thus cannot be sharded.
+* `TIDB_PK_TYPE`: the primary key type of the table. Possible values include `CLUSTERED` (clustered primary key) and `NONCLUSTERED` (non-clustered primary key).
+* `TIDB_PLACEMENT_POLICY_NAME`: the name of the placement policy applied to the table.
+* `TIDB_TABLE_MODE`: the mode of the table, for example, `Normal`, `Import`, or `Restore`.
+* `TIDB_AFFINITY`: the affinity level of the table. It is `table` for non-partitioned tables, `partition` for partitioned tables, and `NULL` when affinity is not enabled.

pd-configuration-file.md

@@ -292,6 +292,13 @@ Configuration items related to scheduling
 + Specifies the upper limit of the `Region Merge` key. When the Region key is greater than the specified value, the PD does not merge the Region with its adjacent Regions.
 + Default value: `540000`. Before v8.4.0, the default value is `200000`. Starting from v8.4.0, the default value is `540000`.
 
+### `max-affinity-merge-region-size` <span class="version-mark">New in v8.5.5</span>
+
++ Controls the threshold for automatically merging small adjacent Regions that belong to the same [affinity](/table-affinity.md) group. When a Region belongs to an affinity group and its size is smaller than this threshold, PD attempts to merge this Region with other small adjacent Regions in the same affinity group to reduce the number of Regions and maintain the affinity effect.
++ Setting it to `0` disables the automatic merging of small adjacent Regions within an affinity group.
++ Default value: `256`
++ Unit: MiB
+
 ### `patrol-region-interval`
 
 + Controls the running frequency at which the checker inspects the health state of a Region. The smaller this value is, the faster the checker runs. Normally, you do not need to adjust this configuration.
@@ -372,6 +379,11 @@ Configuration items related to scheduling
 + The number of the `Region Merge` scheduling tasks performed at the same time. Set this parameter to `0` to disable `Region Merge`.
 + Default value: `8`
 
+### `affinity-schedule-limit` <span class="version-mark">New in v8.5.5</span>
+
++ Controls the number of [affinity](/table-affinity.md) scheduling tasks that can be performed concurrently. Setting it to `0` disables affinity scheduling.
++ Default value: `0`
+
 ### `high-space-ratio`
 
 + The threshold ratio below which the capacity of the store is sufficient. If the space occupancy ratio of the store is smaller than this threshold value, PD ignores the remaining space of the store when performing scheduling, and balances load mainly based on the Region size. This configuration takes effect only when `region-score-formula-version` is set to `v1`.

sql-statements/sql-statement-alter-table.md

@@ -54,6 +54,7 @@ AlterTableSpec ::=
         | TTLEnable EqOpt ( 'ON' | 'OFF' )
         | TTLJobInterval EqOpt stringLit
     )
+|   'AFFINITY' EqOpt stringLit
 |   PlacementPolicyOption
 
 PlacementPolicyOption ::=
@@ -181,6 +182,8 @@ The following major restrictions apply to `ALTER TABLE` in TiDB:
 
 - Changes of some data types (for example, some TIME, Bit, Set, Enum, and JSON types) are not supported due to the compatibility issues of the `CAST` function's behavior between TiDB and MySQL.
 
+- The `AFFINITY` option is a TiDB extension syntax. After `AFFINITY` is enabled for a table, you cannot modify the partition scheme of that table, such as adding, dropping, reorganizing, or swapping partitions. To modify the partition scheme, you must first remove `AFFINITY`.
+
 - Spatial data types are not supported.
 
 - `ALTER TABLE t CACHE | NOCACHE` is a TiDB extension to MySQL syntax. For details, see [Cached Tables](/cached-tables.md).

sql-statements/sql-statement-create-table.md

@@ -117,6 +117,7 @@ TableOption ::=
 |   'UNION' EqOpt '(' TableNameListOpt ')'
 |   'ENCRYPTION' EqOpt EncryptionOpt
 |    'TTL' EqOpt TimeColumnName '+' 'INTERVAL' Expression TimeUnit (TTLEnable EqOpt ( 'ON' | 'OFF' ))? (TTLJobInterval EqOpt stringLit)?
+|   'AFFINITY' EqOpt StringName
 |   PlacementPolicyOption
 
 OnCommitOpt ::=
@@ -168,21 +169,25 @@ The following *table_options* are supported. Other options such as `AVG_ROW_LENG
 |`AUTO_ID_CACHE`| To set the auto ID cache size in a TiDB instance. By default, TiDB automatically changes this size according to allocation speed of auto ID |`AUTO_ID_CACHE` = 200 |
 |`AUTO_RANDOM_BASE`| To set the initial incremental part value of auto_random. This option can be considered as a part of the internal interface. Users can ignore this parameter |`AUTO_RANDOM_BASE` = 0|
 | `CHARACTER SET` | To specify the [character set](/character-set-and-collation.md) for the table | `CHARACTER SET` =  'utf8mb4' |
+| `COLLATE` | To specify the character set collation for the table | `COLLATE` = 'utf8mb4_bin' |
 | `COMMENT` | The comment information | `COMMENT` = 'comment info' |
+| `AFFINITY` | To enable affinity scheduling for a table or partition. It can be set to `'table'` for non-partitioned tables and `'partition'` for partitioned tables. Setting it to `'none'` or leaving it empty disables affinity scheduling. | `AFFINITY` = 'table' |
 
 <CustomContent platform="tidb">
 
 > **Note:**
 >
-> The `split-table` configuration option is enabled by default. When it is enabled, a separate Region is created for each newly created table. For details, see [TiDB configuration file](/tidb-configuration-file.md).
+> - The `split-table` configuration option is enabled by default. When it is enabled, a separate Region is created for each newly created table. For details, see [TiDB configuration file](/tidb-configuration-file.md).
+> - Before using `AFFINITY`, note that modifying the partitioning scheme (such as adding, dropping, reorganizing, or swapping partitions) of a table with affinity enabled is not supported, and configuring `AFFINITY` on temporary tables or views is not supported.
 
 </CustomContent>
 
 <CustomContent platform="tidb-cloud">
 
 > **Note:**
 >
-> TiDB creates a separate Region for each newly created table.
+> - TiDB creates a separate Region for each newly created table.
+> - Before using `AFFINITY`, note that modifying the partitioning scheme (such as adding, dropping, reorganizing, or swapping partitions) of a table with affinity enabled is not supported, and configuring `AFFINITY` on temporary tables or views is not supported.
 
 </CustomContent>
 

sql-statements/sql-statement-show-affinity.md

@@ -0,0 +1,61 @@
+---
+title: SHOW AFFINITY
+summary: An overview of the usage of SHOW AFFINITY for the TiDB database.
+---
+
+# SHOW AFFINITY <span class="version-mark">New in v8.5.5</span>
+
+The `SHOW AFFINITY` statement shows [affinity](/table-affinity.md) scheduling information for tables configured with the `AFFINITY` option, as well as the target replica distribution currently recorded by PD.
+
+## Synopsis
+
+```ebnf+diagram
+ShowAffinityStmt ::=
+    "SHOW" "AFFINITY" ShowLikeOrWhereOpt
+```
+
+`SHOW AFFINITY` supports filtering table names using `LIKE` or `WHERE` clauses.
+
+## Examples
+
+The following examples create two tables with affinity scheduling enabled and show how to view their scheduling information:
+
+```sql
+CREATE TABLE t1 (a INT) AFFINITY = 'table';
+CREATE TABLE tp1 (a INT) AFFINITY = 'partition' PARTITION BY HASH(a) PARTITIONS 2;
+
+SHOW AFFINITY;
+```
+
+The example output is as follows:
+
+```sql
++---------+------------+----------------+-----------------+------------------+----------+--------------+----------------------+
+| Db_name | Table_name | Partition_name | Leader_store_id | Voter_store_ids  | Status   | Region_count | Affinity_region_count|
++---------+------------+----------------+-----------------+------------------+----------+--------------+----------------------+
+| test    | t1         | NULL           | 1               | 1,2,3            | Stable   |            8 |                    8 |
+| test    | tp1        | p0             | 4               | 4,5,6            | Preparing|            4 |                    2 |
+| test    | tp1        | p1             | 4               | 4,5,6            | Preparing|            3 |                    2 |
++---------+------------+----------------+-----------------+------------------+----------+--------------+----------------------+
+```
+
+The meaning of each column is as follows:
+
+- `Leader_store_id`, `Voter_store_ids`: the IDs of TiKV stores recorded by PD, indicating which stores host the target Leader and Voter replicas for the table or partitions. If the target replica locations for the affinity group are not determined, or if [`schedule.affinity-schedule-limit`](/pd-configuration-file.md#affinity-schedule-limit-new-in-v855) is set to `0`, the value is displayed as `NULL`.
+- `Status`: indicates the current status of affinity scheduling. Possible values are:
+    - `Pending`: PD has not started affinity scheduling for the table or partition, such as when Leaders or Voters are not yet determined.
+    - `Preparing`: PD is scheduling Regions to meet affinity requirements.
+    - `Stable`: all Regions have reached the target distribution.
+- `Region_count`: the current number of Regions in the affinity group.
+- `Affinity_region_count`: the number of Regions that currently meet the affinity replica distribution requirements.
+    - When `Affinity_region_count` is less than `Region_count`, it indicates that some Regions have not yet completed replica scheduling based on affinity.
+    - When `Affinity_region_count` equals `Region_count`, it indicates that replica scheduling based on affinity is complete, meaning the distribution of all related Regions meets the affinity requirements. However, this does not indicate that related Region merge operations are complete.
+
+## MySQL compatibility
+
+This statement is a TiDB extension to MySQL syntax.
+
+## See also
+
+- [`CREATE TABLE`](/sql-statements/sql-statement-create-table.md)
+- [`ALTER TABLE`](/sql-statements/sql-statement-alter-table.md)