3 changed files with 6 additions and 816 deletions
--- a/DevOps/Clickhouse/Cluster/0-Bases-and-query.md
+++ b/DevOps/Clickhouse/Cluster/0-Bases-and-query.md
@ -1,164 +0,0 @@
 ---
 title: 0. Основы и запросы
 description: 
 published: true
 date: 2025-05-15T20:02:37.600Z
 tags: clickhouse
 editor: markdown
 dateCreated: 2025-05-15T20:01:45.285Z
 ---
 ## Структура кластера
 ### Найдено на просторах инета:
 ##### Выбор решения:
 - Если **диск загружен на запись (INSERT)** → **шардирование (Distributed)**.  
 - Если **диск загружен на чтение (SELECT)** → **репликация (Replicated)**.  
 - Если **один сервер, но много дисков** → **JBOD или RAID 0/10**.  
 #### Реализуем такую структуру тестового кластера:
 - сервер clickhouse-1
 - сервер clickhouse-2
 - сервер clickhouse-keeper
 Для того, чтобы это было надежно и работало быстро с записью в БД, необходимо сделать репликацию (Replicated) таблиц и распределенный (Distributed) доступ к таблицам.
 Если делать несколько шардов (shard) в настройках кластера, то это значит, что часть данных будет находится на одном сервере, другая часть на другом. При выходе из строя сервера мы потеряем всю базу.
 Поэтому настраиваем shard1 на обоих серверах, а также  replica1  и replica2 для дублирования данных на серверах.
 ### Репликация поддерживается только для таблиц семейства MergeTree:
 https://clickhouse.com/docs/ru/engines/table-engines/mergetree-family/replication
 - ReplicatedMergeTree
 - ReplicatedSummingMergeTree
 - ReplicatedReplacingMergeTree
 - ReplicatedAggregatingMergeTree
 - ReplicatedCollapsingMergeTree
 - ReplicatedVersionedCollapsingMergeTree
 - ReplicatedGraphiteMergeTree
 ### Для одиночного сервера и кластера
 #### Получение списка всех баз данных на сервере
 ```sql
 SHOW DATABASES;
 ```
 #### Просмотр всех таблиц в конкретной базе default
 ```sql
 SHOW TABLES FROM default;
 ```
 #### Получить запрос на создание таблицы из существующей
 ```sql
 show create table default.event;
 ```
 #### Создание базы данных
 ```sql
 CREATE DATABASE IF NOT EXISTS my_database ENGINE = Atomic;
 ```
 Создает новую базу данных с именем `my_database`. Если такая база уже существует, ничего не произойдет благодаря условию `IF NOT EXISTS`.
 Параметр `ENGINE = Atomic` используется для новых версий ClickHouse и позволяет создавать транзакционно безопасные базы данных. В ранних версиях использовался простой синтаксис без указания движка.
 #### Создание таблицы
 ```sql
 CREATE TABLE my_table (
    event_date Date,
    user_id UInt32,
    page_views Int32,
    revenue Float64
 )
 ENGINE = MergeTree()
 ORDER BY (event_date);
 ```
 ##### Вставка данных
 ```sql
 INSERT INTO my_table VALUES ('2023-08-01', 1, 10, 100.5), ('2023-08-02', 2, 15, 150.75);
 ```
 ##### Выборка данных
 ```sql
 SELECT * FROM my_table WHERE event_date >= '2023-08-01' AND event_date <= '2023-08-31';
 ```
 #### Показ структуры таблицы
 ```sql
 DESCRIBE TABLE my_database.my_table;
 ```
 или подробнее:
 ```sql
 SELECT * FROM system.columns WHERE table='my_table' AND database='my_database';
 ```
 ### Узнать сколько занимают места базы
 ```sql
 SELECT 
    database,
    formatReadableSize(sum(bytes_on_disk)) AS total_size
 FROM system.parts
 WHERE active
 GROUP BY database
 ORDER BY sum(bytes_on_disk) DESC;
 ```
 #### Узнать сколько занимают места таблицы в базе
 default
 ```sql
 SELECT
    name AS table_name,
    formatReadableSize(total_bytes) AS size_readable,
    total_rows AS rows_count
 FROM system.tables
 WHERE database = 'default'
 ORDER BY total_bytes DESC;
 ```
 system
 ```sql
 SELECT
    name AS table_name,
    formatReadableSize(total_bytes) AS size_readable,
    total_rows AS rows_count
 FROM system.tables
 WHERE database = 'system'
 ORDER BY total_bytes DESC;
 ```
 #### Очистка базы event_replicated
 ```sql
 TRUNCATE TABLE default.event_replicated;
 ```
 #### Очистка старых данных от определенной даты
 ```sql
 ALTER TABLE my_table DELETE WHERE event_date < '2023-08-01';
 ```
 #### Безопасное удаление таблицы
 ```sql
 DROP TABLE IF EXISTS my_database.my_table;
 ```
 Безопасно удаляет указанную таблицу, если она существует.
 ### Для кластера
 #### Вывести список серверов в кластере 
 ```sql
 SELECT hostName() FROM clusterAllReplicas('smvu2_cluster', 'system', 'one')
 ```
 #### Получение списка Replicated таблиц
 ```sql
 SELECT 
    database,
    name AS table,
    engine
 FROM system.tables
 WHERE engine LIKE 'Replicated%';
 ```
 #### Получение списка Distributed таблиц
 ```sql
 SELECT
    database,
    name AS table,
    engine
 FROM system.tables
 WHERE engine LIKE 'Distributed%';
 ```
 #### Проверить распределение данных базы event на серверах кластера
 ```sql
 SELECT 
    hostName() AS host,
    count() AS count
 FROM clusterAllReplicas(smvu2_cluster, default.event)
 GROUP BY host;
 ```
 #### Удалить таблицу в кластере
 ```sql
 DROP TABLE event_replicated ON CLUSTER smvu2_cluster SYNC;
 ```
--- a/DevOps/Clickhouse/Cluster/1-Install-Clickhouse-cluster.md
+++ b/DevOps/Clickhouse/Cluster/1-Install-Clickhouse-cluster.md
@ -2,14 +2,14 @@
 title: 1. Создание кластера Clickhouse
 description: 
 published: true
-date: 2025-05-15T20:11:12.215Z
+date: 2025-05-15T18:04:57.423Z
 tags: clickhouse, cluster
 editor: markdown
 dateCreated: 2025-05-15T18:04:57.423Z
 ---
 https://clickhouse.com/docs/guides/sre/keeper/clickhouse-keeper
-## Установка и настройка clickhouse, clickhouse-keeper
+
 ### Установка clickhouse-keeper
 ```bash
 apt-get install -y apt-transport-https ca-certificates curl gnupg
@ -20,7 +20,7 @@ apt-get update
 apt-get install -y clickhouse-keeper
 ```
-#### Готовим конфиг clickhouse-keeper
+### Готовим конфиг clickhouse-keeper
 ```
 nano  /etc/clickhouse-keeper/keeper_config.xml
 ```
@ -67,7 +67,7 @@ nano  /etc/clickhouse-keeper/keeper_config.xml
 </clickhouse>
 ```
-#### Пароль пользователя
+### Пароль пользователя
 ```
 nano /etc/clickhouse-server/users.d/default-password.xml
 ```
@ -92,7 +92,7 @@ nano /etc/clickhouse-server/users.d/default-password.xml
 ```bash
 nano /etc/clickhouse-server/config.d/remote_servers.xml
 ```
-##### server 1
+#### server 1
 ```xml
 <yandex>
    <display_name>smvu2_cluster node 1</display_name>
@ -136,7 +136,7 @@ nano /etc/clickhouse-server/config.d/shard.xml
 </yandex>
 ```
-##### server 2
+#### server 2
 ```xml
 <yandex>
    <display_name>smvu2_cluster node 2</display_name>
--- a/DevOps/Clickhouse/Cluster/3-Create-manual.md
+++ b/DevOps/Clickhouse/Cluster/3-Create-manual.md
@ -1,646 +0,0 @@
 ---
 title: 3. Создание таблиц вручную
 description: 
 published: true
 date: 2025-05-15T20:07:34.738Z
 tags: clickhouse
 editor: markdown
 dateCreated: 2025-05-15T20:07:34.738Z
 ---
 Можно создать все таблицы для кластера вручную. 
 Запрашиваем структуру каждой таблицы, создаем под другим именем и копируем данные с исходника.
 ##### Может возникнуть проблема, что таблицы содержат много данных, поэтому внизу статьи будет решение
 ## Создаем реплицированные таблицы
 **Разберем первую таблицу подробно**
 ### 1. api_request_log_replicated
 Запрашиваем структуру
 ```sql
 show create table default.api_request_log
 ```
 Выход:
 ```sql
 CREATE TABLE default.api_request_log
 (
    `uuid` UUID DEFAULT '00000000-0000-0000-0000-000000000000',
    `ver` UInt8,
    `request_ts` DateTime64(3) DEFAULT '0000000000.000',
    `duration` Float64,
    `endpoint` String,
    `method` FixedString(5),
    `status_code` UInt32,
    `remote_addr` String,
    `username` Nullable(String),
    `error` Nullable(String),
    `request_body` Nullable(String) TTL toDateTime(request_ts) + toIntervalMonth(1),
    `query_params` Nullable(String) TTL toDateTime(request_ts) + toIntervalMonth(1),
    `response_body` Nullable(String) TTL toDateTime(request_ts) + toIntervalMonth(1)
 )
 ENGINE = MergeTree(ver)
 PARTITION BY toYYYYMM(request_ts)
 ORDER BY uuid
 SETTINGS index_granularity = 8192;
 ```
 Дорабатываем запрос, чтобы получилась на выходе реплицируемая таблица
 ```sql
 CREATE TABLE default.api_request_log_replicated ON CLUSTER smvu2_cluster
 (
    `uuid` UUID DEFAULT '00000000-0000-0000-0000-000000000000',
    `ver` UInt8,
    `request_ts` DateTime64(3) DEFAULT '0000000000.000',
    `duration` Float64,
    `endpoint` String,
    `method` FixedString(5),
    `status_code` UInt32,
    `remote_addr` String,
    `username` Nullable(String),
    `error` Nullable(String),
    `request_body` Nullable(String) TTL toDateTime(request_ts) + toIntervalMonth(1),
    `query_params` Nullable(String) TTL toDateTime(request_ts) + toIntervalMonth(1),
    `response_body` Nullable(String) TTL toDateTime(request_ts) + toIntervalMonth(1)
 )
 ENGINE = ReplicatedReplacingMergeTree(
    '/clickhouse/tables/{shard}/default/api_request_log_replicated',
    '{replica}',
    ver
 )
 PARTITION BY toYYYYMM(request_ts)
 ORDER BY uuid
 SETTINGS index_granularity = 8192;
 ```
 Копируем данные
 ```sql
 INSERT INTO default.api_request_log_replicated
 SELECT * FROM default.api_request_log
 ```
 ### 2. event_replicated
 ```sql
 show create table default.event
 ```
 ```sql
 CREATE TABLE default.event_replicated ON CLUSTER smvu2_cluster
 (
    `event_id` UUID,
    `unipoll_id` UUID,
    `sensor_id` UUID,
    `sensor_type_id` UUID,
    `sensor_type_name` String,
    `sensor_activation_status_name` String,
    `sensor_logic_status_id` UUID,
    `sensor_logic_status_name` String,
    `sensor_comment` Nullable(String),
    `command` String,
    `priority_name` String,
    `pub_ts` DateTime,
    `sub_ts` DateTime64(3, 'Europe/Moscow'),
    `state_id` UInt8,
    `state_name` String,
    `status_id` UInt8,
    `status_name` String,
    `unitype_id` UInt16,
    `unitype_name` String,
    `system_id` UInt16,
    `system_short_name` String,
    `system_name` String,
    `checked` UInt8,
    `object_id` UUID,
    `object_type_name` String,
    `data_id` UInt8,
    `value` Nullable(String),
    `raw_value` Nullable(String),
    `data` String,
    `region_id` UUID,
    `region_name` String,
    `complex_id` UUID,
    `complex_name` String,
    `collector_id` Nullable(UUID),
    `collector_name` Nullable(String),
    `control_room_id` Nullable(UUID),
    `control_room_name` Nullable(String),
    `picket_id` Nullable(UUID),
    `picket_name` Nullable(String),
    `picket_n` Nullable(UInt32),
    `cstate_id` UUID,
    `cstate_name` String,
    `cstate_color_id` UInt8,
    `sub_ms` UInt64,
    `linked_channel_state` Nullable(String),
    `on_guard` Nullable(UInt8),
    `internal_id` Nullable(String),
    `floor_number` Nullable(Int32)
 )
 ENGINE = ReplicatedMergeTree(
    '/clickhouse/tables/{shard}/default/event_replicated',
    '{replica}'
 )
 PARTITION BY toYYYYMM(pub_ts)
 ORDER BY (priority_name, sensor_type_id, cstate_id, object_id, sensor_id, pub_ts, sub_ts, event_id)
 SETTINGS index_granularity = 8192;
 ```
 Копируем данные
 ```sql
 INSERT INTO default.event_replicated
 SELECT * FROM default.event
 ```
 ### 3. event_automation_replicated
 ```sql
 show create table default.event_automation
 ```
 ```sql
 CREATE TABLE default.event_automation_replicated ON CLUSTER smvu2_cluster
 (
    `event_id` UUID DEFAULT '00000000-0000-0000-0000-000000000000',
    `unipoll_id` UUID DEFAULT '00000000-0000-0000-0000-000000000000',
    `automation_id` UUID DEFAULT '00000000-0000-0000-0000-000000000000',
    `automation_name` String,
    `automation_mode` UInt8,
    `pub_ts` DateTime DEFAULT '0000000000',
    `sub_ts` DateTime64(3) DEFAULT '0000000000.000',
    `sub_ms` UInt64,
    `state_id` UInt8,
    `state_name` String,
    `status_id` UInt8,
    `unitype_id` UInt8,
    `unitype_name` String,
    `group_id` UUID DEFAULT '00000000-0000-0000-0000-000000000000',
    `group_name` String,
    `data` String,
    `cstate_id` UUID DEFAULT '00000000-0000-0000-0000-000000000000',
    `cstate_name` String,
    `cstate_color_id` UInt8
 )
 ENGINE = ReplicatedMergeTree(
    '/clickhouse/tables/{shard}/default/event_automation_replicated',
    '{replica}'
 )
 PARTITION BY toYYYYMMDD(pub_ts)
 ORDER BY (event_id, pub_ts, sub_ts)
 SETTINGS index_granularity = 8192;
 ```
 Копируем данные
 ```sql
 INSERT INTO default.event_automation_replicated
 SELECT * FROM default.event_automation
 ```
 ### 4. event_confirmation_replicated
 ```sql
 show create table default.event_confirmation
 ```
 ```sql
 CREATE TABLE default.event_confirmation_replicated ON CLUSTER smvu2_cluster
 (
    `uuid` UUID,
    `event_id` UUID,
    `reason` String DEFAULT '',
    `event_checked` Int8,
    `username` String,
    `datetime` DateTime,
    `confirmation_group` String,
    `node_id` UUID
 )
 ENGINE = ReplicatedMergeTree(
    '/clickhouse/tables/{shard}/default/event_confirmation_replicated',
    '{replica}'
 )
 PARTITION BY toYYYYMM(datetime)
 ORDER BY datetime
 SETTINGS index_granularity = 8192;
 ```
 Копируем данные
 ```sql
 INSERT INTO default.event_confirmation_replicated
 SELECT * FROM default.event_confirmation
 ```
 ### 5. event_objects_replicated
 ```sql
 show create table default.event_objects
 ```
 ```sql
 CREATE TABLE default.event_objects_replicated ON CLUSTER smvu2_cluster
 (
    `event_id` UUID,
    `node_id` UUID,
    `pub_ts` DateTime,
    `sub_ts` DateTime64(3, 'UTC'),
    `cstate_id` UUID,
    `source_id` UUID,
    `source_name` String DEFAULT '',
    `source_type_id` UUID,
    `event_data` String,
    `complex_id` UUID
 )
 ENGINE = ReplicatedMergeTree(
    '/clickhouse/tables/{shard}/default/event_objects_replicated',
    '{replica}'
 )
 PARTITION BY toYYYYMM(pub_ts)
 PRIMARY KEY event_id
 ORDER BY (event_id, pub_ts, sub_ts)
 SETTINGS index_granularity = 8192;
 ```
 Копируем данные
 ```sql
 INSERT INTO default.event_objects_replicated
 SELECT * FROM default.event_objects
 ```
 ### 6. infi_clickhouse_orm_migrations_replicated
 ```sql
 show create table default.infi_clickhouse_orm_migrations
 ```
 ```sql
 CREATE TABLE default.infi_clickhouse_orm_migrations_replicated ON CLUSTER smvu2_cluster
 (
    `package_name` String,
    `module_name` String,
    `applied` Date DEFAULT '1970-01-01'
 )
 ENGINE = ReplicatedMergeTree(
    '/clickhouse/tables/{shard}/default/infi_clickhouse_orm_migrations_replicated',
    '{replica}'
 )
 PARTITION BY toYYYYMM(applied)
 ORDER BY (package_name, module_name)
 SETTINGS index_granularity = 8192;
 ```
 Копируем данные
 ```sql
 INSERT INTO default.infi_clickhouse_orm_migrations_replicated
 SELECT * FROM default.infi_clickhouse_orm_migrations
 ```
 ### 7. unchecked_event_replicated
 ```sql
 show create table default.unchecked_event
 ```
 ```sql
 CREATE TABLE default.unchecked_event_replicated ON CLUSTER smvu2_cluster
 (
    `event_id` UUID,
    `unipoll_id` Nullable(UUID),
    `sensor_id` Nullable(UUID),
    `sensor_type_id` Nullable(UUID),
    `sensor_type_name` Nullable(String),
    `sensor_activation_status_name` Nullable(String),
    `sensor_logic_status_id` Nullable(UUID),
    `sensor_logic_status_name` Nullable(String),
    `sensor_comment` Nullable(String),
    `command` Nullable(String),
    `priority_name` Nullable(String),
    `pub_ts` Nullable(DateTime),
    `sub_ts` DateTime64(3),
    `state_id` Nullable(UInt8),
    `state_name` Nullable(String),
    `status_id` Nullable(UInt8),
    `status_name` Nullable(String),
    `unitype_id` Nullable(UInt16),
    `unitype_name` Nullable(String),
    `system_id` Nullable(UInt16),
    `system_short_name` Nullable(String),
    `system_name` Nullable(String),
    `checked` Nullable(UInt8),
    `object_id` Nullable(UUID),
    `object_type_name` Nullable(String),
    `data_id` Nullable(UInt8),
    `data` Nullable(String),
    `region_id` Nullable(UUID),
    `region_name` Nullable(String),
    `complex_id` Nullable(UUID),
    `complex_name` Nullable(String),
    `collector_id` Nullable(UUID),
    `collector_name` Nullable(String),
    `control_room_id` Nullable(UUID),
    `control_room_name` Nullable(String),
    `picket_id` Nullable(UUID),
    `picket_name` Nullable(String),
    `picket_n` Nullable(UInt32),
    `cstate_id` Nullable(UUID),
    `cstate_name` Nullable(String),
    `cstate_color_id` Nullable(UInt8),
    `sub_ms` Nullable(UInt64),
    `linked_channel_state` Nullable(String),
    `on_guard` Nullable(UInt8),
    `internal_id` Nullable(String),
    `floor_number` Nullable(Int32),
    `sign` Int8,
    `value` Nullable(String),
    `raw_value` Nullable(String)
 )
 ENGINE = ReplicatedCollapsingMergeTree(
    '/clickhouse/tables/{shard}/default/unchecked_event_replicated',
    '{replica}',
    sign
    )
 PARTITION BY toYYYYMM(sub_ts)
 PRIMARY KEY event_id
 ORDER BY event_id
 SETTINGS index_granularity = 8192;
 ```
 Копируем
 ```sql
 INSERT INTO default.unchecked_event_replicated
 SELECT * FROM default.unchecked_event;
 ```
 ### 8. unchecked_objects_event_replicated
 ```sql
 show create table default.unchecked_objects_event;
 ```
 ```sql
 CREATE TABLE default.unchecked_objects_event_replicated ON CLUSTER smvu2_cluster
 (
    `event_id` UUID,
    `node_id` Nullable(UUID),
    `complex_id` Nullable(UUID),
    `pub_ts` Nullable(DateTime),
    `sub_ts` DateTime64(3, 'UTC'),
    `cstate_id` Nullable(UUID),
    `source_id` Nullable(UUID),
    `source_name` Nullable(String),
    `source_type_id` Nullable(UUID),
    `event_data` Nullable(String),
    `sign` Int8
 )
 ENGINE = ReplicatedCollapsingMergeTree(
    '/clickhouse/tables/{shard}/default/unchecked_objects_event_replicated',
    '{replica}',
    sign
 )
 PARTITION BY toYYYYMM(sub_ts)
 PRIMARY KEY event_id
 ORDER BY event_id
 SETTINGS index_granularity = 8192;
 ```
 Копируем
 ```sql
 INSERT INTO default.unchecked_objects_event_replicated
 SELECT * FROM default.unchecked_objects_event;
 ```
 ### Проверка состояния репликации после создания:
 ```sql
 SELECT database, table, is_leader, total_replicas, active_replicas
  FROM system.replicas
  WHERE database = 'default'
 ```
 ### Узнаем и сравниваем сколько занимают места таблицы в базе default
 ```sql
 SELECT
    name AS table_name,
    formatReadableSize(total_bytes) AS size_readable,
    total_rows AS rows_count
 FROM system.tables
 WHERE database = 'default'
 ORDER BY total_bytes DESC;
 ```
 ### Переименовываем исходные таблицы
 ```sql
 rename table api_request_log to api_request_log_old;
 rename table event to event_old;
 rename table event_objects to event_objects_old;
 rename table event_confirmation to event_confirmation_old;
 rename table audit_new to audit_new_old;
 rename table event_automation to event_automation_old;
 rename table infi_clickhouse_orm_migrations to infi_clickhouse_orm_migrations_old;
 rename table unchecked_event to unchecked_event_old;
 rename table unchecked_objects_event to unchecked_objects_event_old;
 ```
 ### Создаем Distributed-таблицы
 ```sql
 CREATE TABLE default.event ON CLUSTER smvu2_cluster AS default.event_replicated 
 ENGINE = Distributed(smvu2_cluster, default, api_request_log_replicated, rand());
 ```
 ```sql
 CREATE TABLE default.event ON CLUSTER smvu2_cluster AS default.event_replicated 
 ENGINE = Distributed(smvu2_cluster, default, event_replicated, rand());
 ```
 ```sql
 CREATE TABLE default.audit_new ON CLUSTER smvu2_cluster AS default.audit_new_replicated 
 ENGINE = Distributed(smvu2_cluster, default, audit_new_replicated, rand());
 ```
 ```sql
 CREATE TABLE default.event_objects ON CLUSTER smvu2_cluster AS default.event_objects_replicated 
 ENGINE = Distributed(smvu2_cluster, default, event_objects_replicated, rand());
 ```
 ```sql
 CREATE TABLE default.event_automation ON CLUSTER smvu2_cluster AS default.event_automation_replicated 
 ENGINE = Distributed(smvu2_cluster, default, event_automation_replicated, rand());
 ```
 ```sql
 CREATE TABLE default.event_confirmation ON CLUSTER smvu2_cluster AS default.event_confirmation_replicated 
 ENGINE = Distributed(smvu2_cluster, default, event_confirmation_replicated, rand());
 ```
 ```sql
 CREATE TABLE default.infi_clickhouse_orm_migrations ON CLUSTER smvu2_cluster AS default.infi_clickhouse_orm_migrations_replicated 
 ENGINE = Distributed(smvu2_cluster, default, infi_clickhouse_orm_migrations_replicated, rand());
 ```
 ```sql
 CREATE TABLE default.unchecked_event ON CLUSTER smvu2_cluster AS default.unchecked_event_replicated 
 ENGINE = Distributed(smvu2_cluster, default, unchecked_event_replicated, rand());
 ```
 ```sql
 CREATE TABLE default.unchecked_objects_event ON CLUSTER smvu2_cluster AS default.unchecked_objects_event_replicated 
 ENGINE = Distributed(smvu2_cluster, default, unchecked_objects_event_replicated, rand());
 ```
 ### Решение проблемы размера таблицы
 #### 1. Используя ограничение отправляемых строк, прикладываю скрипт, который делает в автоматическом режиме
 ```bash
 #!/bin/bash
 # Список таблиц для репликации
 TABLES=(
  api_request_log
  audit_new
  event
  event_automation
  event_confirmation
  event_objects
  infi_clickhouse_orm_migrations
 )
 PASSWORD="q1"
 BATCH_SIZE=100000  # Размер порции для вставки
 MAX_THREADS=4      # Максимальное количество потоков для вставки
 for table in "${TABLES[@]}"; do
  replicated_table="${table}_replicated"
  echo "Processing $table to $replicated_table"
  # 1. Получаем общее количество строк
  total_rows=$(clickhouse-client --password "$PASSWORD" --query="
    SELECT count() FROM remote('localhost', 'default', '$table')")
  echo "Total rows to copy: $total_rows"
  # 2. Копируем данные порциями
  for ((offset=0; offset<total_rows; offset+=BATCH_SIZE)); do
    echo "Copying rows $((offset+1))-$((offset+BATCH_SIZE))..."
    clickhouse-client --password "$PASSWORD" --query="
      INSERT INTO default.$replicated_table
      SELECT * FROM remote('localhost', 'default', '$table')
      LIMIT $BATCH_SIZE OFFSET $offset
      SETTINGS
        max_insert_threads=$MAX_THREADS,
        max_memory_usage='10Gi'"
    if [ $? -ne 0 ]; then
      echo "Error copying batch $((offset/BATCH_SIZE+1))"
      break
    fi
  done
  # 3. Проверяем количество скопированных строк
  copied_rows=$(clickhouse-client --password "$PASSWORD" --query="
    SELECT count() FROM default.$replicated_table")
  echo "Successfully copied $copied_rows/$total_rows rows from $table to $replicated_table"
 done
 ```
 #### 2. Копирование с помощью программы clickhouse-copier
 Создается конфиг подключения к кластеру (keeper)
 config.xml
 ```xml
 <yandex>
    <logger>
        <level>warning</level>
        <log>/var/log/clickhouse-server/clickhouse-copier.log</log>
        <errorlog>/var/log/clickhouse-server/clickhouse-copier.err.log</errorlog>
    </logger>
    <zookeeper>
        <!-- Для ClickHouse Keeper (рекомендуется) -->
        <node>
            <host>172.16.212.41</host>
            <port>9181</port>
        </node>
    </zookeeper>
    <distributed_ddl>
        <path>/clickhouse/task_queue/ddl</path>
    </distributed_ddl>
    <http_port>9999</http_port>
 </yandex>
 ```
 Создается папка tasks и там размещаются файлы для каждой таблицы, например api_request_log.xml
 ```xml
 <yandex>
    <!-- Configuration of clusters as in an ordinary server config -->
    <remote_servers>
        <source_cluster>
            <shard>
                <internal_replication>true</internal_replication>
                    <replica>
                        <host>172.16.212.14</host>
                        <port>9000</port>
                        <user>default</user>
                        <password>q1</password>
                        <secure>0</secure>
                    </replica>
            </shard>
        </source_cluster>
        <destination_cluster>
            <shard>
                <internal_replication>true</internal_replication>
                    <replica>
                        <host>172.16.212.14</host>
                        <port>9000</port>
                        <user>default</user>
                        <password>q1</password>
                        <secure>0</secure>
                    </replica>
            </shard>
        </destination_cluster>
    </remote_servers>
    <!-- How many simultaneously active workers are possible. If you run more workers superfluous workers will sleep. -->
    <max_workers>2</max_workers>
    <!-- Setting used to fetch (pull) data from source cluster tables -->
    <settings_pull>
        <readonly>1</readonly>
    </settings_pull>
    <!-- Setting used to insert (push) data to destination cluster tables -->
    <settings_push>
        <readonly>0</readonly>
    </settings_push>
    <!-- Common setting for fetch (pull) and insert (push) operations. Also, copier process context uses it.
         They are overlaid by <settings_pull/> and <settings_push/> respectively. -->
    <settings>
        <connect_timeout>3</connect_timeout>
        <!-- Sync insert is set forcibly, leave it here just in case. -->
        <insert_distributed_sync>1</insert_distributed_sync>
    </settings>
    <tables>
        <!-- A table task, copies one table. -->
        <event>
            <!-- Source cluster name (from <remote_servers/> section) and tables in it that should be copied -->
            <cluster_pull>source_cluster</cluster_pull>
            <database_pull>default</database_pull>
            <table_pull>api_request_log</table_pull>
            <!-- Destination cluster name and tables in which the data should be inserted -->
            <cluster_push>destination_cluster</cluster_push>
            <database_push>default</database_push>
            <table_push>api_request_log_replicated</table_push>
            <engine>
                ENGINE = ReplicatedReplacingMergeTree(
                    '/clickhouse/tables/{shard}/default/api_request_log_replicated',
                    '{replica}',
                    ver
                )
                PARTITION BY toYYYYMM(request_ts)
                ORDER BY uuid
                SETTINGS index_granularity = 8192;
            </engine>
            <sharding_key>rand()</sharding_key>
        </event>
    </tables>
 </yandex>
 ```
 В каждом файле меняется название базы в разделе **tables**
 После этого запускается копирование, где --task-path должен быть уникальным для каждой таблицы
 ```bash
 clickhouse-copier --config config.xml --task-file ./tasks/api_request_log.xml --task-path /clickhouse/copier/task1
 ```