flume agent组件

0 前提:

地址: http://flume.apache.org/FlumeUserGuide.html

版本: flume1.6.0

只摘抄对自己有用的

1  flume 对avro的支持:

RPC

An Avro client included in the Flume distribution can send a given file to Flume Avro source using avro RPC mechanism:

$ bin/flume-ng avro-client -H localhost -p 41414 -F /usr/logs/log.10

The above command will send the contents of /usr/logs/log.10 to to the Flume source listening on that ports.

source:  avro, thrift, jms(mq), kafka, http, custom
channel: memory, jdbc, kafka, file,
sink: hdfs, hive(不用于生产), logger, avro, thrift, null, hbase, elastrcsearch, kafka, custom
interceptor: Host, Timestamp, Static , UUID,  Regex Filtering

2