本文共 2203 字，大约阅读时间需要 7 分钟。
Flume容错机制用于实现故障转移，即在一台机器出现故障时，另一台机器代替其工作。这种设计通常用于解决单点故障问题，通过为关键部件设置备份，确保数据不会丢失或处理中断。
容错机制的工作原理是：当主机器出现故障时，容错入口会自动切换到备用机器，确保流数据继续正常传输。备份的数量越多，容错能力越强，但同时也会带来资源的浪费。
本节将介绍如何配置一个简单的Flume容错场景。
# agent1配置agent1.sources = r1agent1.channels = c1agent1.sinks = k1 k2agent1.sinkgroups = g1agent1.channels.c1.type = memoryagent1.channels.c1.capacity = 1000agent1.channels.c1.transactionCapacity = 100agent1.sources.r1.channels = c1agent1.sources.r1.type = execagent1.sources.r1.command = tail -F /home/xiaokang/logs/456.log#basic sono配置agent1.sinks.k1.channel = c1agent1.sinks.k1.type = avroagent1.sinks.k1.hostname = hadoop02agent1.sinks.k1.port = 52020agent1.sinks.k2.channel = c1agent1.sinks.k2.type = avroagent1.sinks.k2.hostname = hadoop03agent1.sinks.k2.port = 52020# failover配置agent1.sinkgroups.g1 Processor.type = failoveragent1.sinkgroups.g1.sinks = k1 k2agent1.sinkgroups.g1.processor.priority.k1 = 10agent1.sinkgroups.g1.processor.priority.k2 = 1agent1.sinkgroups.g1.processor.maxPenalty = 10000
a1.sources = r1a1.sinks = k1a1.channels = c1a1.sources.r1.type = avroa1.sources.r1.bind = hadoop02a1.sources.r1.port = 52020a1.sinks.k1.type = loggera1.sinks.k1.channel = c1
a1.sources = r1a1.sinks = k1a1.channels = c1a1.sources.r1.type = avroa1.sources.r1.bind = hadoop02a1.sources.r1.port = 52020a1.sinks.k1.type = loggera1.sinks.k1.channel = c1
# hadoop01启动$ flume-ng agent -n agent1 -c /opt/software/flume-1.9.0/conf/ -f flume_scripts/exec-avro-failover.properties -Dflume.root.logger=INFO,console# hadoop02启动$ flume-ng agent -n a1 -c /opt/software/flume-1.9.0/conf/ -f flume_scripts/avro-logger-failover.properties -Dflume.root.logger=INFO,console# hadoop03启动$ flume-ng agent -n a1 -c /opt/software/flume-1.9.0/conf/ -f flume_scripts/avro-logger-failover.properties -Dflume.root.logger=INFO,console
在hadoop01执行以下命令，生成模拟日志数据：
$ while true; do date >> /home/xiaokang/logs/456.log; done
启动Flume Agent后，监控hadoop02的日志接收情况。通过登录hadoop02查看Flume运行日志，确认数据是否 正常流向hadoop03（备用端）。
转载地址：http://ijpaz.baihongyu.com/