support for dividing table for internal storage

[calvin0327]

What type of PR is this?
/kind feature

What this PR does / why we need it:

Which issue(s) this PR fixes:
Fixes #601

Special notes for your reviewer:

Does this PR introduce a user-facing change?:

support for dividing table for internal storage

[clusterpedia-bot]

Hi @calvin0327,
Thanks for your pull request!
If the PR is ready, use the /auto-cc command to assign Reviewer to Review.
We will review it shortly.

Details

Instructions for interacting with me using comments are available here.
If you have questions or suggestions related to my behavior, please file an issue against the gh-ci-bot repository.

[calvin0327]

I'll test the results later.

[calvin0327]

The test results:

1. Create pediacluster cr:

apiVersion: cluster.clusterpedia.io/v1alpha2
kind: PediaCluster
metadata:
  name: cluster-example
spec:
  apiserver: "https://10.30.43.43:6443"
  caData: 
  tokenData:
  certData: 
  keyData: 
  syncResources:
    - group: apps
      resources:
        - deployments
    - group: ""
      resources:
        - pods

2. After runing clustersynchro-manager component and generate these tables in clusterpedia database:

[calvin0327]

@Iceber The pr basic feature have been completed, can we have reveiw first?

[theseaofstars]

1. we do not support DivisionPolicyCustom in this pr?
2. what if the divisionpolicy is "custom" or other value?

[theseaofstars]

if func CleanClusterResource is called when controller is not in dividing table mode. Calling generatetableFor may be incorrect.

[calvin0327]

/retest

[Iceber]

How to resolve table deletions is something we'll need to consider later.

[calvin0327]

/retest

[calvin0327]

here are test results:

apiVersion: cluster.clusterpedia.io/v1alpha2
kind: PediaCluster
metadata:
  name: cluster-example
spec:
  apiserver: "https://10.30.43.43:6443"
  caData:
  tokenData:
  certData:
  keyData:
  syncResources:
    - group: apps
      resources:
        - deployments
    - group: ""
      resources:
        - pods

[root@master01 ~]# kubectl  get pediacluster
NAME              READY   VERSION   APISERVER
cluster-example   True    v1.29.2   https://10.6.212.13:6443
[root@master01 ~]#

# DivisionPolicy is "None"

mysql> show tables;
+------------------------+
| Tables_in_clusterpedia |
+------------------------+
| resources              |
+------------------------+
1 row in set (0.01 sec)
mysql>

[root@master01 ~]# kubectl --cluster clusterpedia get deploy -A
NAMESPACE             CLUSTER           NAME                                  READY   UP-TO-DATE   AVAILABLE   AGE
clusterpedia-system   cluster-example   clusterpedia-apiserver                1/1     1            1           3h17m
clusterpedia-system   cluster-example   clusterpedia-clustersynchro-manager   1/1     1            1           3h17m
clusterpedia-system   cluster-example   clusterpedia-controller-manager       1/1     1            1           3h17m
kube-system           cluster-example   cilium-operator                       2/2     2            2           77d
kube-system           cluster-example   coredns                               2/2     2            2           84d
kubeflow              cluster-example   training-operator                     1/1     1            1           24d
volcano-system        cluster-example   volcano-admission                     1/1     1            1           30d
volcano-system        cluster-example   volcano-controllers                   1/1     1            1           30d
volcano-system        cluster-example   volcano-scheduler                     1/1     1            1           30d

[root@master01 ~]# kubectl delete pediacluster cluster-example
pediacluster.cluster.clusterpedia.io "cluster-example" deleted

[root@master01 ~]# kubectl --cluster clusterpedia get deploy -A
Error from server (NotFound): Unable to list "apps/v1, Resource=deployments": the server could not find the requested resource (get deployments.apps)

# DivisionPolicy is "GVR"

mysql> show tables;
+------------------------+
| Tables_in_clusterpedia |
+------------------------+
| apps_v1_deployments    |
| v1_pods                |
+------------------------+
2 rows in set (0.00 sec)

mysql>

[root@master01 clusterpedia]# kubectl --cluster clusterpedia get deploy -A
NAMESPACE             CLUSTER           NAME                                  READY   UP-TO-DATE   AVAILABLE   AGE
clusterpedia-system   cluster-example   clusterpedia-apiserver                1/1     1            1           3h57m
clusterpedia-system   cluster-example   clusterpedia-clustersynchro-manager   1/1     1            1           3h57m
clusterpedia-system   cluster-example   clusterpedia-controller-manager       1/1     1            1           3h57m
kube-system           cluster-example   cilium-operator                       2/2     2            2           77d
kube-system           cluster-example   coredns                               2/2     2            2           84d
kubeflow              cluster-example   training-operator                     1/1     1            1           24d
volcano-system        cluster-example   volcano-admission                     1/1     1            1           30d
volcano-system        cluster-example   volcano-controllers                   1/1     1            1           30d
volcano-system        cluster-example   volcano-scheduler                     1/1     1            1           30d

[root@master01 clusterpedia]# kubectl delete pediacluster cluster-example
pediacluster.cluster.clusterpedia.io "cluster-example" deleted

[root@master01 clusterpedia]# kubectl --cluster clusterpedia get deploy -A
Error from server (NotFound): Unable to list "apps/v1, Resource=deployments": the server could not find the requested resource (get deployments.apps)