kafka 机架感知配置测试

文档:

Balancing Replicas Across Racks

The rack awareness feature spreads replicas of the same partition across different racks. This extends the guarantees Kafka provides for broker-failure to cover rack-failure, limiting the risk of data loss should all the brokers on a rack fail at once. The feature can also be applied to other broker groupings such as availability zones in EC2.

You can specify that a broker belongs to a particular rack by adding a property to the broker config:
1
broker.rack=my-rack-id
When a topic is created, modified or replicas are redistributed, the rack constraint will be honoured, ensuring replicas span as many racks as they can (a partition will span min(#racks, replication-factor) different racks).

The algorithm used to assign replicas to brokers ensures that the number of leaders per broker will be constant, regardless of how brokers are distributed across racks. This ensures balanced throughput.

However if racks are assigned different numbers of brokers, the assignment of replicas will not be even. Racks with fewer brokers will get more replicas, meaning they will use more storage and put more resources into replication. Hence it is sensible to configure an equal number of brokers per rack. 

相关参数:
broker.rackRack of the broker. This will be used in rack aware replication assignment for fault tolerance. Examples: RACK1, us-east-1dstringnullmedium

==================================================================================

一、测试方案

1、节点: 172.28.1.114  3        broker.rack=yanjiao
                 172.28.1.88    4        broker.rack=yanjiao
                 172.28.1.87    5        broker.rack=ucloud
                 172.28.1.89    6        broker.rack=ucloud
注意:不同机架位的节点数应保持平衡,否则负载分布将发生倾斜,节点数较少的机架位节点会获得更多的负载;(参加分配策略)
2、以 topic zs_invoice测试
     {"topics": [{"topic": "zs_invoice"}],
  "version":1
    }

3、生成分配策略:
 bin/kafka-reassign-partitions.sh --zookeeper 172.28.1.89:2181 --topics-to-move-json-file topics-to-move.json --broker-list "3,4,5,6" --generate
注意:此时生成分配策略时候,会先依次对broker.id进行排序(机架感知配置生效)ex 3,5,4,6  ,再此基础上在进行分配策略;确保partions副本分布在不同的机架位;

4、执行分配策略
bin/kafka-reassign-partitions.sh --zookeeper 172.28.1.89:2181 --reassignment-json-file expand-cluster-reassignment.json --execute -throttle 60000000
5、确认完成并移除限速
bin/kafka-reassign-partitions.sh --zookeeper 172.28.1.89:2181 --reassignment-json-file expand-cluster-reassignment.json --verify

Pingbacks已打开。

Trackbacks

引用地址

评论
  1. PierreHor PierreHor 发布于 2021年4月18日 01:48 #

    https://paxilst.com/# paxil for depression

  2. EvaGal EvaGal 发布于 2021年4月18日 02:29 #

    <a href="http://mpcialis.com/">tadalafil generic in india</a>

  3. SamGal SamGal 发布于 2021年4月18日 03:05 #

    <a href="http://azzmeds.com/">tadalafil price south africa</a> <a href="http://cialissx.com/">cialis 40 mg online</a> <a href="http://cialiszm.com/">cialis buy no prescription</a> <a href="http://viagraops.com/">order sildenafil tablets</a> <a href="http://viagraztab.com/">generic viagra online in usa</a>

  4. KimGal KimGal 发布于 2021年4月18日 04:00 #

    <a href="http://viagrapoint.com/">360 viagra</a>

  5. LisaGal LisaGal 发布于 2021年4月18日 04:10 #

    <a href="https://viagraxpill.com/">buy viagra online discount</a>

  6. Berrypat Berrypat 发布于 2021年4月18日 04:44 #

    Find out <a href="https://istrippers.pro/?s=scarletbenz" target="_blank" title="scarletbenz">scarletbenz nude</a> for free :)

  7. KimGal KimGal 发布于 2021年4月18日 06:14 #

    <a href="http://fnnpharm.com/">buy tadalafil tablets 20 mg</a>

  8. JimGal JimGal 发布于 2021年4月18日 06:44 #

    <a href="http://cialiseti.com/">cialis 800mg</a> <a href="http://viagraxl.com/">kamagra paypal</a> <a href="http://mpcialis.com/">cialis 2 mg</a> <a href="http://555pharmacy.com/">pharmacy websites</a> <a href="http://cialisvm.com/">cheap online cialis</a>

  9. JoeGal JoeGal 发布于 2021年4月18日 06:45 #

    <a href="http://ipillviagra.com/">viagra cream buy online</a> <a href="http://pharmacyrp.com/">all in one pharmacy</a> <a href="http://mpcialis.com/">tadalafil for women</a> <a href="http://cialissx.com/">dapoxetine online buy</a>

  10. Loan Cash Loan Cash 发布于 2021年4月18日 07:08 #

    <a href="http://baslending.com/">online installment loans</a>

  11. Stripaccix Stripaccix 发布于 2021年4月18日 07:40 #

    Discover the best <a href="https://ivirtualgirls.com/?s=kitystar" target="_blank" title="kitystar">kitystar naked</a> ever for free!

  12. otc cialis otc cialis 发布于 2021年4月18日 07:47 #

    cheap cialis generic online http://himscanada.com/ - buy cialis

  13. KimGal KimGal 发布于 2021年4月18日 10:31 #

    <a href="http://viagrafpack.com/">brand viagra 100mg</a>

  14. IvyGal IvyGal 发布于 2021年4月18日 11:52 #

    <a href="http://yestablets.com/">lexapro price in india</a> <a href="http://lmopharm.com/">amoxicillin 500mg capsule price</a> <a href="http://viagrabtc.com/">sildenafil 80 mg</a> <a href="http://rxphealth.com/">sildenafil nz</a> <a href="http://viagracap.com/">brand viagra online canadian pharmacy</a> <a href="http://fnnpharm.com/">tadalafil cost usa</a> <a href="http://cialissx.com/">brand cialis best price</a> <a href="http://atcialis.com/">buy cialis using paypal</a> <a href="http://viagrahere.com/">can you order viagra for woman</a> <a href="http://lacialis.com/">tadalafil in india online</a>

  15. SamGal SamGal 发布于 2021年4月18日 11:57 #

    <a href="http://ordcialis.com/">buy cialis online in india</a> <a href="http://lmopharm.com/">augmentin 93</a> <a href="http://crspharmacy.com/">canadian pharmacy india</a>

  16. Payday Loans Online Payday Loans Online 发布于 2021年4月18日 12:53 #

    <a href="https://clevlending.com/">loan interest rates</a>

  17. EvaGal EvaGal 发布于 2021年4月18日 13:15 #

    <a href="http://tlstabs.com/">metformin order online</a>

  18. JaneGal JaneGal 发布于 2021年4月18日 14:33 #

    <a href="http://iviagracialislevitra.com/">buy viagra canada fast shipping</a>

发表评论