一、source
source 类型有exec、spooldir、netcat、avro、http
支持的类型可看：org.apache.flume.conf.source.SourceType
1.exec
创建配置文件 flume-exec-memory-logger.conf
app1.sources = r1
app1.channels = c1
app1.sinks = k1
app1.sources.r1.type = exec
app1.sources.r1.command = tail -F /tmp/mysource/t.txt
app1.channels.c1.type = memory
app1.sinks.k1.type = logger
app1.sources.r1.channels = c1
app1.sinks.k1.channel = c1启动：
bin/flume-ng agent --conf conf --conf-file conf-file/flume-exec-memory-logger.conf --name app1 -Dflume.root.logger=INFO,console
发送数据：
向文件追加数据echo "dd" >> /tmp/mysource/t.txt
可以看到控制台日志的输出
2.spooldir
创建配置文件 flume-spooldir-memory-logger.conf
app2.sources = r1
app2.channels = c1
app2.sinks = k1
app2.sources.r1.type = spooldir
app2.sources.r1.spoolDir = /tmp/mydata
app2.channels.c1.type = memory
app2.sinks.k1.type = logger
app2.sources.r1.channels = c1
app2.sinks.k1.channel = c1启动：
bin/flume-ng agent --conf conf --conf-file conf-file/flume-spooldir-memory-logger.conf --name app2 -Dflume.root.logger=INFO,console
发送数据：
把数据文件移动到/tmp/mydata
可以看到控制台日志的输出
3.netcat
创建配置文件 flume-netcat-memory-logger.conf
app3.sources = r1
app3.channels = c1
app3.sinks = k1
app3.sources.r1.type = netcat
app3.sources.r1.bind = 0.0.0.0
app3.sources.r1.port = 44444
app3.channels.c1.type = memory
app3.sinks.k1.type = logger
app3.sources.r1.channels = c1
app3.sinks.k1.channel = c1启动：
bin/flume-ng agent --conf conf --conf-file conf-file/flume-netcat-memory-logger.conf --name app3 -Dflume.root.logger=INFO,console
发送数据：
nc localhost 44444
可以看到控制台日志的输出
4.avro
创建配置文件 flume-avro-memory-logger.conf
app4.sources = r1
app4.channels = c1
app4.sinks = k1
app4.sources.r1.type = avro
app4.sources.r1.bind = 0.0.0.0
app4.sources.r1.port = 44444
app4.channels.c1.type = memory
app4.sinks.k1.type = logger
app4.sources.r1.channels = c1
app4.sinks.k1.channel = c1启动：
bin/flume-ng agent --conf conf --conf-file conf-file/flume-avro-memory-logger.conf --name app4 -Dflume.root.logger=INFO,console
发送数据：
bin/flume-ng avro-client --conf conf --host 0.0.0.0 --port 44444 --filename a.txt
可以看到控制台日志的输出
5.http
创建配置文件 flume-http-memory-logger.conf
app5.sources = r1
app5.channels = c1
app5.sinks = k1
app5.sources.r1.type = http
app5.sources.r1.port = 44444
app5.channels.c1.type = memory
app5.sinks.k1.type = logger
app5.sources.r1.channels = c1
app5.sinks.k1.channel = c1启动：
bin/flume-ng agent --conf conf --conf-file conf-file/flume-http-memory-logger.conf --name app5 -Dflume.root.logger=INFO,console
发送数据：
curl -X POST -d '[{ "headers" :{"a":"a1", "b":"b1"}, "body" : "hello http flume"}]' http://localhost:44444
可以看到控制台日志的输出
二、channel
channel 类型有memory、file
支持的类型可看：org.apache.flume.conf.channel.ChannelType
1.memory
参看 flume-http-memory-logger.conf
2.file
创建配置文件 flume-http-file-logger.conf
app2.sources = r1
app2.channels = c1
app2.sinks = k1
app2.sources.r1.type = http
app2.sources.r1.port = 44444
app2.channels.c1.type = file
app2.channels.c1.checkpointDir = /tmp/mychannel/checkpoint
app2.channels.c1.dataDirs = /tmp/mychannel/data
app2.sinks.k1.type = logger
app2.sources.r1.channels = c1
app2.sinks.k1.channel = c1启动：
bin/flume-ng agent --conf conf --conf-file conf-file/flume-http-file-logger.conf --name app2 -Dflume.root.logger=INFO,console
发送数据：
curl -X POST -d '[{ "headers" :{"a":"a1", "b":"b1"}, "body" : "hello http flume"}]' http://localhost:44444
可以看到控制台日志的输出
三、sink
sink类型有logger、file_roll
支持的类型可看：org.apache.flume.conf.sink.SinkType
1.logger
参看 flume-http-memory-logger.conf
2.file_roll
创建配置文件 flume-http-memory-file.conf
app2.sources = r1
app2.channels = c1
app2.sinks = k1
app2.sources.r1.type = http
app2.sources.r1.port = 44444
app2.channels.c1.type = memory
app2.sinks.k1.type = file_roll
app2.sinks.k1.sink.directory = /tmp/mysink
app2.sources.r1.channels = c1
app2.sinks.k1.channel = c1启动：
bin/flume-ng agent --conf conf --conf-file conf-file/flume-http-memory-file.conf --name app2 -Dflume.root.logger=INFO,console
发送数据：
curl -X POST -d '[{ "headers" :{"a":"a1", "b":"b1"}, "body" : "hello http flume"}]' http://localhost:44444
可以在/tmp/mysink看到输出的文件
