Added cleanup meachnism for tendrl setup by shtripat · Pull Request #129 · Tendrl/tendrl-ansible

shtripat · 2018-11-14T06:05:43Z

The cleanup playbook performs the below steps

Stops collectd, tendrl-node-agent and tendrl-gluster-integration
on storage nodes
Stops tendrl-node-agent, tendrl-monitoring-integration, tendrl-notifier,
tendrl-api on tendrl server node
Stops grafana-server, carbon-cache and etcd services on storage nodes
Cleans up etcd, grafana and carbon data files

tendrl-bug-id: /issues/63
Signed-off-by: Shubhendu shtripat@redhat.com

shtripat · 2018-11-14T06:06:09Z

@nthomas-redhat @mbukatov @GowthamShanmugam @anmolsachan @cloudbehl please review

GowthamShanmugam · 2018-11-14T06:31:55Z

cleanup.yml

+  roles:
+    - tendrl-server-cleanup
+
+- hosts: gluster_servers


I feel stopping storage nodes services first and then stop server is safer

Done in next update.

The cleanup playbook performs the below steps - Stops collectd, tendrl-node-agent and tendrl-gluster-integration on storage nodes - Stops tendrl-node-agent, tendrl-monitoring-integration, tendrl-notifier, tendrl-api on tendrl server node - Stops grafana-server, carbon-cache and etcd services on storage nodes - Cleans up etcd, grafana and carbon data files tendrl-bug-id: Tendrl/issues/63 Signed-off-by: Shubhendu <shtripat@redhat.com>

GowthamShanmugam · 2018-11-14T06:57:17Z

LGTM

cloudbehl · 2018-11-14T07:57:22Z

roles/tendrl-ansible.tendrl-server-cleanup/main.yml

+    name: httpd
+    state: stopped
+
+- name: Cleanup etcd data


isn't it destructive to remove the ETCD? User may have different namespace running for different things can we just delete the data?

Doing it for the dev purpose makes sense to me but doing this same on production server of a client, I am a bit skeptical about it.

thoughts? @shtripat @nthomas-redhat @anmolsachan @GowthamShanmugam

if we are sure then the customer doesn't have anything else running on the tendrl server machine then we can take this path.

We save tendrl data in default profile. That itself is not very correct I feel.
We can have a disclaimer in document regarding the same, that it will flush all the etcd data.

Correct I agree shubhendu. IF some other data in default profile that will also get flushed.

There could be another option to delete all tendrl created dirs under etcd like /clusters, /nodes and tags etc. After this keep etcd running as usual and tendrl-ansible shouldnt cause issues. We will need to test the scenario though.

cloudbehl · 2018-11-14T07:58:51Z

roles/tendrl-ansible.tendrl-server-cleanup/main.yml

+    path: "{{etcd_db_path}}/"
+    state: absent
+
+- name: Cleanup grafana data


same as above

Here as well, I feel we are using default location to save tendrl data.

what if someone is having other dashboards on grafana which are not related to tendrl. By cleaning grafana I think those will be lost too and then there will be no recovery for it.

I think grafanadb file will be common for all

Here as well we can try to remove only tendrl specific dashboards and keep grafana-server running. Need to verify though.

cloudbehl · 2018-11-14T07:58:59Z

roles/tendrl-ansible.tendrl-server-cleanup/main.yml

+    path: "{{grafana_db_file}}"
+    state: absent
+
+- name: Cleanup carbon data


same as above

In case of carbon we will delete only tendrl specific directory and nothing else.

Correct. I missed that.

mbukatov

I'm providing few suggestions/question in comments below.

But main question is, what purpose this cleanup has. Because it cleans up the data, but keeps the configuration files intact, which means that it is not guaranteed that it will be always possible to install tendrl again via tendrl-ansible as if it was done from scratch.

Also make sure that the Travis CI checks are passing (yamlling and ansible-lint).

mbukatov · 2018-12-20T17:55:20Z

README.md

+
+    ```
+    $ ansible-playbook -i inventory_file cleanup.yml
+    ```


Maybe I would not repeat all the details already mentioned in the installation section to avoid duplication a bit, and make this little more succinct (after all, one needs to install first to do the cleanup). We could point out that the same ideas as for installation apply.

mbukatov · 2018-12-20T17:57:05Z

cleanup.yml

+    grafana_db_file: "/var/lib/grafana/grafana.db"
+
+    # carbon data to be deleted
+    carbon_data_path: "/var/lib/carbon/whisper/tendrl/"


I would rather moved these default values into roles/tendrl-ansible.tendrl-server-cleanup/defaults/main.yml.

mbukatov · 2018-12-20T18:03:53Z

README.md


+## How do I cleanup Tendrl setup using tendrl-ansible?
+
+1)  If you use tendrl-ansible from rpm package, copy `cleanup.yml` playbook into


Main information we are missing here is what does this cleanup mean and why would one want to do one. This helps to convey context of this feature.

Also it seems that the cleanup deletes all tendrl internal data (etcd, graphite, ...). It seems to me that this needs to be highlighted here again.

mbukatov · 2018-12-20T18:06:51Z

roles/tendrl-ansible.tendrl-server-cleanup/main.yml

+- name: Stop tendrl services
+  service:
+    name: tendrl-node-agent
+    state: stopped


Why we are not stopping tendrl-api?

Ack. Will add tendrl-api as well.

shtripat requested a review from mbukatov as a code owner November 14, 2018 06:05

GowthamShanmugam reviewed Nov 14, 2018

View reviewed changes

shtripat force-pushed the cleanup branch from 267a1b9 to 82778a6 Compare November 14, 2018 06:53

cloudbehl reviewed Nov 14, 2018

View reviewed changes

mbukatov suggested changes Dec 20, 2018

View reviewed changes


		## How do I cleanup Tendrl setup using tendrl-ansible?

		1) If you use tendrl-ansible from rpm package, copy `cleanup.yml` playbook into

Conversation

shtripat commented Nov 14, 2018

Uh oh!

shtripat commented Nov 14, 2018

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

GowthamShanmugam commented Nov 14, 2018

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

cloudbehl Nov 14, 2018 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

mbukatov left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

cloudbehl Nov 14, 2018 •

edited

Loading