'Big Data Platform/Storm' 카테고리의 글 목록

Big Data Platform/Storm

[에러] Address already in use (Bind failed) 2018.03.21
Storm 0.9.6 설치 및 환경 구축 2018.03.19

[에러] Address already in use (Bind failed)

2018. 3. 21. 10:48

storm 0.9.6의 ui 서비스 실행 중에 다음과 같은 에러 발생 시

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
$ ./bin/storm ui &
[1] 7306
Running: /usr/lib/jvm/java-8-oracle/bin/java -server -Dstorm.options= -Dstorm.home=/usr/local/apache-storm-0.9.6 -Dstorm.log.dir=/usr/local/apache-storm-0.9.6/logs -Djava.library.path=/usr/local/lib:/opt/local/lib:/usr/lib -Dstorm.conf.file= -cp /usr/local/apache-storm-0.9.6/lib/clj-time-0.4.1.jar:/usr/local/apache-storm-0.9.6/lib/tools.cli-0.2.4.jar:/usr/local/apache-storm-0.9.6/lib/commons-lang-2.5.jar:/usr/local/apache-storm-0.9.6/lib/tools.macro-0.1.0.jar:/usr/local/apache-storm-0.9.6/lib/joda-time-2.0.jar:/usr/local/apache-storm-0.9.6/lib/tools.logging-0.2.3.jar:/usr/local/apache-storm-0.9.6/lib/objenesis-1.2.jar:/usr/local/apache-storm-0.9.6/lib/commons-exec-1.1.jar:/usr/local/apache-storm-0.9.6/lib/ring-core-1.1.5.jar:/usr/local/apache-storm-0.9.6/lib/jetty-util-6.1.26.jar:/usr/local/apache-storm-0.9.6/lib/chill-java-0.3.5.jar:/usr/local/apache-storm-0.9.6/lib/json-simple-1.1.jar:/usr/local/apache-storm-0.9.6/lib/jgrapht-core-0.9.0.jar:/usr/local/apache-storm-0.9.6/lib/log4j-over-slf4j-1.6.6.jar:/usr/local/apache-storm-0.9.6/lib/logback-classic-1.0.13.jar:/usr/local/apache-storm-0.9.6/lib/ring-devel-0.3.11.jar:/usr/local/apache-storm-0.9.6/lib/logback-core-1.0.13.jar:/usr/local/apache-storm-0.9.6/lib/commons-logging-1.1.3.jar:/usr/local/apache-storm-0.9.6/lib/disruptor-2.10.4.jar:/usr/local/apache-storm-0.9.6/lib/compojure-1.1.3.jar:/usr/local/apache-storm-0.9.6/lib/asm-4.0.jar:/usr/local/apache-storm-0.9.6/lib/math.numeric-tower-0.0.1.jar:/usr/local/apache-storm-0.9.6/lib/commons-io-2.4.jar:/usr/local/apache-storm-0.9.6/lib/carbonite-1.4.0.jar:/usr/local/apache-storm-0.9.6/lib/core.incubator-0.1.0.jar:/usr/local/apache-storm-0.9.6/lib/reflectasm-1.07-shaded.jar:/usr/local/apache-storm-0.9.6/lib/snakeyaml-1.11.jar:/usr/local/apache-storm-0.9.6/lib/commons-codec-1.6.jar:/usr/local/apache-storm-0.9.6/lib/minlog-1.2.jar:/usr/local/apache-storm-0.9.6/lib/commons-fileupload-1.2.1.jar:/usr/local/apache-storm-0.9.6/lib/hiccup-0.3.6.jar:/usr/local/apache-storm-0.9.6/lib/ring-servlet-0.3.11.jar:/usr/local/apache-storm-0.9.6/lib/clout-1.0.1.jar:/usr/local/apache-storm-0.9.6/lib/ring-jetty-adapter-0.3.11.jar:/usr/local/apache-storm-0.9.6/lib/servlet-api-2.5.jar:/usr/local/apache-storm-0.9.6/lib/storm-core-0.9.6.jar:/usr/local/apache-storm-0.9.6/lib/jetty-6.1.26.jar:/usr/local/apache-storm-0.9.6/lib/jline-2.11.jar:/usr/local/apache-storm-0.9.6/lib/slf4j-api-1.7.5.jar:/usr/local/apache-storm-0.9.6/lib/kryo-2.21.jar:/usr/local/apache-storm-0.9.6/lib/clj-stacktrace-0.2.2.jar:/usr/local/apache-storm-0.9.6/lib/clojure-1.5.1.jar:/usr/local/apache-storm-0.9.6:/usr/local/apache-storm-0.9.6/conf -Xmx768m -Dlogfile.name=ui.log -Dlogback.configurationFile=/usr/local/apache-storm-0.9.6/logback/cluster.xml backtype.storm.ui.core
Exception in thread "main" java.net.BindException: Address already in use (Bind failed)
        at java.net.PlainSocketImpl.socketBind(Native Method)
        at java.net.AbstractPlainSocketImpl.bind(AbstractPlainSocketImpl.java:387)
        at java.net.ServerSocket.bind(ServerSocket.java:375)
        at java.net.ServerSocket.<init>(ServerSocket.java:237)
        at java.net.ServerSocket.<init>(ServerSocket.java:181)
        at org.mortbay.jetty.bio.SocketConnector.newServerSocket(SocketConnector.java:80)
        at org.mortbay.jetty.bio.SocketConnector.open(SocketConnector.java:73)
        at org.mortbay.jetty.AbstractConnector.doStart(AbstractConnector.java:283)
        at org.mortbay.jetty.bio.SocketConnector.doStart(SocketConnector.java:147)
        at org.mortbay.component.AbstractLifeCycle.start(AbstractLifeCycle.java:50)
        at org.mortbay.jetty.Server.doStart(Server.java:235)
        at org.mortbay.component.AbstractLifeCycle.start(AbstractLifeCycle.java:50)
        at ring.adapter.jetty$run_jetty.invoke(jetty.clj:65)
        at backtype.storm.ui.core$start_server_BANG_.invoke(core.clj:920)
        at backtype.storm.ui.core$_main.invoke(core.clj:923)
        at clojure.lang.AFn.applyToHelper(AFn.java:159)
        at clojure.lang.AFn.applyTo(AFn.java:151)
        at backtype.storm.ui.core.main(Unknown Source)
 
Colored by Color Scripter
cs

default ui 포트가 사용 중이므로 발생할 가능성이 높으므로 storm 홈 디렉터리 이하 conf 디렉터리에서 storm.yaml 파일을 편집 후 재실행합니다.

ui 포트의 기본값은 8080입니다.

1. storm.yaml 편집

1
2
$ vi ./conf/storm.yaml
 
cs

2. storm.yaml의 하단에 다음의 내용을 추가 후 저장 및 종료합니다.

1
2
ui.port: 8087
 
cs

3. ui 서비스 실행

1
2
3
4
$ ./bin/storm ui &
[2] 7600
Running: /usr/lib/jvm/java-8-oracle/bin/java -server -Dstorm.options= -Dstorm.home=/usr/local/apache-storm-0.9.6 -Dstorm.log.dir=/usr/local/apache-storm-0.9.6/logs -Djava.library.path=/usr/local/lib:/opt/local/lib:/usr/lib -Dstorm.conf.file= -cp /usr/local/apache-storm-0.9.6/lib/clj-time-0.4.1.jar:/usr/local/apache-storm-0.9.6/lib/tools.cli-0.2.4.jar:/usr/local/apache-storm-0.9.6/lib/commons-lang-2.5.jar:/usr/local/apache-storm-0.9.6/lib/tools.macro-0.1.0.jar:/usr/local/apache-storm-0.9.6/lib/joda-time-2.0.jar:/usr/local/apache-storm-0.9.6/lib/tools.logging-0.2.3.jar:/usr/local/apache-storm-0.9.6/lib/objenesis-1.2.jar:/usr/local/apache-storm-0.9.6/lib/commons-exec-1.1.jar:/usr/local/apache-storm-0.9.6/lib/ring-core-1.1.5.jar:/usr/local/apache-storm-0.9.6/lib/jetty-util-6.1.26.jar:/usr/local/apache-storm-0.9.6/lib/chill-java-0.3.5.jar:/usr/local/apache-storm-0.9.6/lib/json-simple-1.1.jar:/usr/local/apache-storm-0.9.6/lib/jgrapht-core-0.9.0.jar:/usr/local/apache-storm-0.9.6/lib/log4j-over-slf4j-1.6.6.jar:/usr/local/apache-storm-0.9.6/lib/logback-classic-1.0.13.jar:/usr/local/apache-storm-0.9.6/lib/ring-devel-0.3.11.jar:/usr/local/apache-storm-0.9.6/lib/logback-core-1.0.13.jar:/usr/local/apache-storm-0.9.6/lib/commons-logging-1.1.3.jar:/usr/local/apache-storm-0.9.6/lib/disruptor-2.10.4.jar:/usr/local/apache-storm-0.9.6/lib/compojure-1.1.3.jar:/usr/local/apache-storm-0.9.6/lib/asm-4.0.jar:/usr/local/apache-storm-0.9.6/lib/math.numeric-tower-0.0.1.jar:/usr/local/apache-storm-0.9.6/lib/commons-io-2.4.jar:/usr/local/apache-storm-0.9.6/lib/carbonite-1.4.0.jar:/usr/local/apache-storm-0.9.6/lib/core.incubator-0.1.0.jar:/usr/local/apache-storm-0.9.6/lib/reflectasm-1.07-shaded.jar:/usr/local/apache-storm-0.9.6/lib/snakeyaml-1.11.jar:/usr/local/apache-storm-0.9.6/lib/commons-codec-1.6.jar:/usr/local/apache-storm-0.9.6/lib/minlog-1.2.jar:/usr/local/apache-storm-0.9.6/lib/commons-fileupload-1.2.1.jar:/usr/local/apache-storm-0.9.6/lib/hiccup-0.3.6.jar:/usr/local/apache-storm-0.9.6/lib/ring-servlet-0.3.11.jar:/usr/local/apache-storm-0.9.6/lib/clout-1.0.1.jar:/usr/local/apache-storm-0.9.6/lib/ring-jetty-adapter-0.3.11.jar:/usr/local/apache-storm-0.9.6/lib/servlet-api-2.5.jar:/usr/local/apache-storm-0.9.6/lib/storm-core-0.9.6.jar:/usr/local/apache-storm-0.9.6/lib/jetty-6.1.26.jar:/usr/local/apache-storm-0.9.6/lib/jline-2.11.jar:/usr/local/apache-storm-0.9.6/lib/slf4j-api-1.7.5.jar:/usr/local/apache-storm-0.9.6/lib/kryo-2.21.jar:/usr/local/apache-storm-0.9.6/lib/clj-stacktrace-0.2.2.jar:/usr/local/apache-storm-0.9.6/lib/clojure-1.5.1.jar:/usr/local/apache-storm-0.9.6:/usr/local/apache-storm-0.9.6/conf -Xmx768m -Dlogfile.name=ui.log -Dlogback.configurationFile=/usr/local/apache-storm-0.9.6/logback/cluster.xml backtype.storm.ui.core
 
Colored by Color Scripter
cs

* &는 백그라운드로 서비스를 실행하라는 의미입니다.

저작자표시

'Big Data Platform > Storm' 카테고리의 다른 글

Storm 0.9.6 설치 및 환경 구축 (0)	2018.03.19

Storm 0.9.6 설치 및 환경 구축

2018. 3. 19. 18:33

* 사전 설치 요구 사항

- zeromq

- jzmq

1. wget을 통해 storm 압축 파일을 내려받습니다.

1
2
3
4
5
6
7
8
9
10
11
12
$ wget https://archive.apache.org/dist/storm/apache-storm-0.9.6/apache-storm-0.9.6.tar.gz
--2018-03-19 16:47:31--  https://archive.apache.org/dist/storm/apache-storm-0.9.6/apache-storm-0.9.6.tar.gz
Resolving archive.apache.org (archive.apache.org)... 163.172.17.199
Connecting to archive.apache.org (archive.apache.org)|163.172.17.199|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 21010966 (20M) [application/x-gzip]
Saving to: ‘apache-storm-0.9.6.tar.gz’
 
apache-storm-0.9.6.tar.gz     100%[==============================================>]  20.04M  1.36MB/s    in 19s
 
2018-03-19 16:47:51 (1.07 MB/s) - ‘apache-storm-0.9.6.tar.gz’ saved [21010966/21010966]
 
Colored by Color Scripter
cs

2. 해당 압축 파일을 해제합니다.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
$ tar -xvzf ./apache-storm-0.9.6.tar.gz
apache-storm-0.9.6/examples/storm-starter/storm-starter-topologies-0.9.6.jar
apache-storm-0.9.6/conf/storm.yaml
apache-storm-0.9.6/conf/storm_env.ini
apache-storm-0.9.6/RELEASE
apache-storm-0.9.6/logback/cluster.xml
apache-storm-0.9.6/LICENSE
apache-storm-0.9.6/NOTICE
apache-storm-0.9.6/README.markdown
apache-storm-0.9.6/DISCLAIMER
apache-storm-0.9.6/CHANGELOG.md
apache-storm-0.9.6/SECURITY.md
apache-storm-0.9.6/lib/storm-core-0.9.6.jar
apache-storm-0.9.6/lib/clojure-1.5.1.jar
apache-storm-0.9.6/lib/clj-time-0.4.1.jar
apache-storm-0.9.6/lib/joda-time-2.0.jar
apache-storm-0.9.6/lib/compojure-1.1.3.jar
apache-storm-0.9.6/lib/core.incubator-0.1.0.jar
apache-storm-0.9.6/lib/tools.macro-0.1.0.jar
apache-storm-0.9.6/lib/clout-1.0.1.jar
apache-storm-0.9.6/lib/ring-core-1.1.5.jar
apache-storm-0.9.6/lib/commons-codec-1.6.jar
apache-storm-0.9.6/lib/commons-io-2.4.jar
apache-storm-0.9.6/lib/commons-fileupload-1.2.1.jar
apache-storm-0.9.6/lib/servlet-api-2.5.jar
apache-storm-0.9.6/lib/hiccup-0.3.6.jar
apache-storm-0.9.6/lib/ring-devel-0.3.11.jar
apache-storm-0.9.6/lib/clj-stacktrace-0.2.2.jar
apache-storm-0.9.6/lib/ring-jetty-adapter-0.3.11.jar
apache-storm-0.9.6/lib/ring-servlet-0.3.11.jar
apache-storm-0.9.6/lib/jetty-6.1.26.jar
apache-storm-0.9.6/lib/jetty-util-6.1.26.jar
apache-storm-0.9.6/lib/tools.logging-0.2.3.jar
apache-storm-0.9.6/lib/math.numeric-tower-0.0.1.jar
apache-storm-0.9.6/lib/tools.cli-0.2.4.jar
apache-storm-0.9.6/lib/commons-exec-1.1.jar
apache-storm-0.9.6/lib/commons-lang-2.5.jar
apache-storm-0.9.6/lib/json-simple-1.1.jar
apache-storm-0.9.6/lib/carbonite-1.4.0.jar
apache-storm-0.9.6/lib/kryo-2.21.jar
apache-storm-0.9.6/lib/reflectasm-1.07-shaded.jar
apache-storm-0.9.6/lib/asm-4.0.jar
apache-storm-0.9.6/lib/minlog-1.2.jar
apache-storm-0.9.6/lib/objenesis-1.2.jar
apache-storm-0.9.6/lib/chill-java-0.3.5.jar
apache-storm-0.9.6/lib/snakeyaml-1.11.jar
apache-storm-0.9.6/lib/commons-logging-1.1.3.jar
apache-storm-0.9.6/lib/disruptor-2.10.4.jar
apache-storm-0.9.6/lib/jgrapht-core-0.9.0.jar
apache-storm-0.9.6/lib/logback-classic-1.0.13.jar
apache-storm-0.9.6/lib/logback-core-1.0.13.jar
apache-storm-0.9.6/lib/slf4j-api-1.7.5.jar
apache-storm-0.9.6/lib/log4j-over-slf4j-1.6.6.jar
apache-storm-0.9.6/lib/jline-2.11.jar
apache-storm-0.9.6/bin/storm
apache-storm-0.9.6/bin/storm-config.cmd
apache-storm-0.9.6/bin/storm.cmd
apache-storm-0.9.6/public/css/
apache-storm-0.9.6/public/images/
apache-storm-0.9.6/public/js/
apache-storm-0.9.6/public/templates/
apache-storm-0.9.6/public/component.html
apache-storm-0.9.6/public/css/bootstrap-1.4.0.css
apache-storm-0.9.6/public/css/style.css
apache-storm-0.9.6/public/images/spinner.gif
apache-storm-0.9.6/public/index.html
apache-storm-0.9.6/public/js/arbor-graphics.js
apache-storm-0.9.6/public/js/arbor-tween.js
apache-storm-0.9.6/public/js/arbor.js
apache-storm-0.9.6/public/js/bootstrap-twipsy.js
apache-storm-0.9.6/public/js/jquery-1.6.2.min.js
apache-storm-0.9.6/public/js/jquery.blockUI.min.js
apache-storm-0.9.6/public/js/jquery.cookies.2.2.0.min.js
apache-storm-0.9.6/public/js/jquery.mustache.js
apache-storm-0.9.6/public/js/jquery.tablesorter.min.js
apache-storm-0.9.6/public/js/moment.min.js
apache-storm-0.9.6/public/js/script.js
apache-storm-0.9.6/public/js/url.min.js
apache-storm-0.9.6/public/js/visualization.js
apache-storm-0.9.6/public/templates/component-page-template.html
apache-storm-0.9.6/public/templates/index-page-template.html
apache-storm-0.9.6/public/templates/json-error-template.html
apache-storm-0.9.6/public/templates/topology-page-template.html
apache-storm-0.9.6/public/topology.html
apache-storm-0.9.6/examples/
apache-storm-0.9.6/examples/storm-starter/
apache-storm-0.9.6/examples/storm-starter/multilang/
apache-storm-0.9.6/examples/storm-starter/multilang/resources/
apache-storm-0.9.6/examples/storm-starter/src/
apache-storm-0.9.6/examples/storm-starter/src/clj/
apache-storm-0.9.6/examples/storm-starter/src/clj/storm/
apache-storm-0.9.6/examples/storm-starter/src/clj/storm/starter/
apache-storm-0.9.6/examples/storm-starter/src/clj/storm/starter/clj/
apache-storm-0.9.6/examples/storm-starter/src/jvm/
apache-storm-0.9.6/examples/storm-starter/src/jvm/storm/
apache-storm-0.9.6/examples/storm-starter/src/jvm/storm/starter/
apache-storm-0.9.6/examples/storm-starter/src/jvm/storm/starter/bolt/
apache-storm-0.9.6/examples/storm-starter/src/jvm/storm/starter/spout/
apache-storm-0.9.6/examples/storm-starter/src/jvm/storm/starter/tools/
apache-storm-0.9.6/examples/storm-starter/src/jvm/storm/starter/trident/
apache-storm-0.9.6/examples/storm-starter/src/jvm/storm/starter/util/
apache-storm-0.9.6/examples/storm-starter/test/
apache-storm-0.9.6/examples/storm-starter/test/jvm/
apache-storm-0.9.6/examples/storm-starter/test/jvm/storm/
apache-storm-0.9.6/examples/storm-starter/test/jvm/storm/starter/
apache-storm-0.9.6/examples/storm-starter/test/jvm/storm/starter/bolt/
apache-storm-0.9.6/examples/storm-starter/test/jvm/storm/starter/tools/
apache-storm-0.9.6/examples/storm-starter/multilang/resources/asyncSplitsentence.js
apache-storm-0.9.6/examples/storm-starter/multilang/resources/randomsentence.js
apache-storm-0.9.6/examples/storm-starter/multilang/resources/splitsentence.js
apache-storm-0.9.6/examples/storm-starter/multilang/resources/splitsentence.py
apache-storm-0.9.6/examples/storm-starter/multilang/resources/splitsentence.rb
apache-storm-0.9.6/examples/storm-starter/multilang/resources/storm.js
apache-storm-0.9.6/examples/storm-starter/multilang/resources/storm.py
apache-storm-0.9.6/examples/storm-starter/multilang/resources/storm.rb
apache-storm-0.9.6/examples/storm-starter/pom.xml
apache-storm-0.9.6/examples/storm-starter/README.markdown
apache-storm-0.9.6/examples/storm-starter/src/clj/storm/starter/clj/word_count.clj
apache-storm-0.9.6/examples/storm-starter/src/jvm/storm/starter/BasicDRPCTopology.java
apache-storm-0.9.6/examples/storm-starter/src/jvm/storm/starter/bolt/AbstractRankerBolt.java
apache-storm-0.9.6/examples/storm-starter/src/jvm/storm/starter/bolt/IntermediateRankingsBolt.java
apache-storm-0.9.6/examples/storm-starter/src/jvm/storm/starter/bolt/PrinterBolt.java
apache-storm-0.9.6/examples/storm-starter/src/jvm/storm/starter/bolt/RollingCountBolt.java
apache-storm-0.9.6/examples/storm-starter/src/jvm/storm/starter/bolt/SingleJoinBolt.java
apache-storm-0.9.6/examples/storm-starter/src/jvm/storm/starter/bolt/TotalRankingsBolt.java
apache-storm-0.9.6/examples/storm-starter/src/jvm/storm/starter/ExclamationTopology.java
apache-storm-0.9.6/examples/storm-starter/src/jvm/storm/starter/ManualDRPC.java
apache-storm-0.9.6/examples/storm-starter/src/jvm/storm/starter/PrintSampleStream.java
apache-storm-0.9.6/examples/storm-starter/src/jvm/storm/starter/ReachTopology.java
apache-storm-0.9.6/examples/storm-starter/src/jvm/storm/starter/RollingTopWords.java
apache-storm-0.9.6/examples/storm-starter/src/jvm/storm/starter/SingleJoinExample.java
apache-storm-0.9.6/examples/storm-starter/src/jvm/storm/starter/spout/RandomSentenceSpout.java
apache-storm-0.9.6/examples/storm-starter/src/jvm/storm/starter/spout/TwitterSampleSpout.java
apache-storm-0.9.6/examples/storm-starter/src/jvm/storm/starter/tools/NthLastModifiedTimeTracker.java
apache-storm-0.9.6/examples/storm-starter/src/jvm/storm/starter/tools/Rankable.java
apache-storm-0.9.6/examples/storm-starter/src/jvm/storm/starter/tools/RankableObjectWithFields.java
apache-storm-0.9.6/examples/storm-starter/src/jvm/storm/starter/tools/Rankings.java
apache-storm-0.9.6/examples/storm-starter/src/jvm/storm/starter/tools/SlidingWindowCounter.java
apache-storm-0.9.6/examples/storm-starter/src/jvm/storm/starter/tools/SlotBasedCounter.java
apache-storm-0.9.6/examples/storm-starter/src/jvm/storm/starter/TransactionalGlobalCount.java
apache-storm-0.9.6/examples/storm-starter/src/jvm/storm/starter/TransactionalWords.java
apache-storm-0.9.6/examples/storm-starter/src/jvm/storm/starter/trident/TridentReach.java
apache-storm-0.9.6/examples/storm-starter/src/jvm/storm/starter/trident/TridentWordCount.java
apache-storm-0.9.6/examples/storm-starter/src/jvm/storm/starter/util/StormRunner.java
apache-storm-0.9.6/examples/storm-starter/src/jvm/storm/starter/util/TupleHelpers.java
apache-storm-0.9.6/examples/storm-starter/src/jvm/storm/starter/WordCountTopology.java
apache-storm-0.9.6/examples/storm-starter/src/jvm/storm/starter/WordCountTopologyNode.java
apache-storm-0.9.6/examples/storm-starter/test/jvm/storm/starter/bolt/IntermediateRankingsBoltTest.java
apache-storm-0.9.6/examples/storm-starter/test/jvm/storm/starter/bolt/RollingCountBoltTest.java
apache-storm-0.9.6/examples/storm-starter/test/jvm/storm/starter/bolt/TotalRankingsBoltTest.java
apache-storm-0.9.6/examples/storm-starter/test/jvm/storm/starter/tools/MockTupleHelpers.java
apache-storm-0.9.6/examples/storm-starter/test/jvm/storm/starter/tools/NthLastModifiedTimeTrackerTest.java
apache-storm-0.9.6/examples/storm-starter/test/jvm/storm/starter/tools/RankableObjectWithFieldsTest.java
apache-storm-0.9.6/examples/storm-starter/test/jvm/storm/starter/tools/RankingsTest.java
apache-storm-0.9.6/examples/storm-starter/test/jvm/storm/starter/tools/SlidingWindowCounterTest.java
apache-storm-0.9.6/examples/storm-starter/test/jvm/storm/starter/tools/SlotBasedCounterTest.java
apache-storm-0.9.6/external/storm-kafka/storm-kafka-0.9.6-javadoc.jar
apache-storm-0.9.6/external/storm-kafka/storm-kafka-0.9.6-sources.jar
apache-storm-0.9.6/external/storm-kafka/storm-kafka-0.9.6.jar
apache-storm-0.9.6/external/storm-kafka/README.md
apache-storm-0.9.6/external/storm-hdfs/storm-hdfs-0.9.6-javadoc.jar
apache-storm-0.9.6/external/storm-hdfs/storm-hdfs-0.9.6-sources.jar
apache-storm-0.9.6/external/storm-hdfs/storm-hdfs-0.9.6.jar
apache-storm-0.9.6/external/storm-hdfs/README.md
apache-storm-0.9.6/external/storm-hbase/storm-hbase-0.9.6-javadoc.jar
apache-storm-0.9.6/external/storm-hbase/storm-hbase-0.9.6-sources.jar
apache-storm-0.9.6/external/storm-hbase/storm-hbase-0.9.6.jar
apache-storm-0.9.6/external/storm-hbase/README.md
 
Colored by Color Scripter
cs

3. storm 디렉터리를 적절한 위치로 이동 혹은 복사합니다.

1
2
$ sudo cp -r ./apache-storm-0.9.6 /usr/local/
 
Colored by Color Scripter
cs

4. storm 디렉터리의 소유 권한을 해당 계정에 부여합니다.

1
2
$ sudo chown -R hadoop-user:hadoop ./apache-storm-0.9.6/
 
Colored by Color Scripter
cs

5. storm 디렉터리의 링크를 생성합니다.

1
2
$ sudo ln -sf ./apache-storm-0.9.6 ./storm
 
Colored by Color Scripter
cs

6. 설정 파일 편집

: storm의 홈 디렉터리 이하에 conf 디렉터리

6-1. storm.yaml 편집

1
2
$ vi ./storm.yaml
 
cs

6-2. storm.yaml 변경

: 변경 전

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
# Licensed to the Apache Software Foundation (ASF) under one
# or more contributor license agreements.  See the NOTICE file
# distributed with this work for additional information
# regarding copyright ownership.  The ASF licenses this file
# to you under the Apache License, Version 2.0 (the
# "License"); you may not use this file except in compliance
# with the License.  You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
 
########### These MUST be filled in for a storm configuration
# storm.zookeeper.servers:
#     - "server1"
#     - "server2"
#
# nimbus.host: "nimbus"
#
#
# ##### These may optionally be filled in:
#
## List of custom serializations
# topology.kryo.register:
#     - org.mycompany.MyType
#     - org.mycompany.MyType2: org.mycompany.MyType2Serializer
#
## List of custom kryo decorators
# topology.kryo.decorators:
#     - org.mycompany.MyDecorator
#
## Locations of the drpc servers
# drpc.servers:
#     - "server1"
#     - "server2"
 
## Metrics Consumers
# topology.metrics.consumer.register:
#   - class: "backtype.storm.metric.LoggingMetricsConsumer"
#     parallelism.hint: 1
#   - class: "org.mycompany.MyMetricsConsumer"
#     parallelism.hint: 1
#     argument:
#       - endpoint: "metrics-collector.mycompany.org"
 
Colored by Color Scripter
cs

: 변경 후

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
# Licensed to the Apache Software Foundation (ASF) under one
# or more contributor license agreements.  See the NOTICE file
# distributed with this work for additional information
# regarding copyright ownership.  The ASF licenses this file
# to you under the Apache License, Version 2.0 (the
# "License"); you may not use this file except in compliance
# with the License.  You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
 
########### These MUST be filled in for a storm configuration
storm.zookeeper.servers:
    - master
    - secondary
    - datanode-a
    - datanode-b
 
nimbus.host: "master"
#
#
# ##### These may optionally be filled in:
#
## List of custom serializations
# topology.kryo.register:
#     - org.mycompany.MyType
#     - org.mycompany.MyType2: org.mycompany.MyType2Serializer
#
## List of custom kryo decorators
# topology.kryo.decorators:
#     - org.mycompany.MyDecorator
#
## Locations of the drpc servers
# drpc.servers:
#     - "server1"
#     - "server2"
 
## Metrics Consumers
# topology.metrics.consumer.register:
#   - class: "backtype.storm.metric.LoggingMetricsConsumer"
#     parallelism.hint: 1
#   - class: "org.mycompany.MyMetricsConsumer"
#     parallelism.hint: 1
#     argument:
#       - endpoint: "metrics-collector.mycompany.org"
 
supervisor.slots.ports:
    - 6700
    - 6701
    - 6702
    - 6703
storm.messaging.transport: "backtype.storm.messaging.netty.Context"
storm.messaging.netty.buffer_size: 16384
storm.messaging.netty.max_retries: 10
storm.messaging.netty.min_wait_ms: 1000
storm.messaging.netty.max_wait_ms: 5000
 
Colored by Color Scripter
cs

7. 이전 단계까지의 작업을 진행한 storm 디렉터리를 압축합니다.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
$ sudo tar -cvzf ./apache-storm-0.9.6.tar.gz ./apache-storm-0.9.6
./apache-storm-0.9.6/
./apache-storm-0.9.6/CHANGELOG.md
./apache-storm-0.9.6/examples/
./apache-storm-0.9.6/examples/storm-starter/
./apache-storm-0.9.6/examples/storm-starter/src/
./apache-storm-0.9.6/examples/storm-starter/src/clj/
./apache-storm-0.9.6/examples/storm-starter/src/clj/storm/
./apache-storm-0.9.6/examples/storm-starter/src/clj/storm/starter/
./apache-storm-0.9.6/examples/storm-starter/src/clj/storm/starter/clj/
./apache-storm-0.9.6/examples/storm-starter/src/clj/storm/starter/clj/word_count.clj
./apache-storm-0.9.6/examples/storm-starter/src/jvm/
./apache-storm-0.9.6/examples/storm-starter/src/jvm/storm/
./apache-storm-0.9.6/examples/storm-starter/src/jvm/storm/starter/
./apache-storm-0.9.6/examples/storm-starter/src/jvm/storm/starter/TransactionalGlobalCount.java
./apache-storm-0.9.6/examples/storm-starter/src/jvm/storm/starter/WordCountTopology.java
./apache-storm-0.9.6/examples/storm-starter/src/jvm/storm/starter/spout/
./apache-storm-0.9.6/examples/storm-starter/src/jvm/storm/starter/spout/RandomSentenceSpout.java
./apache-storm-0.9.6/examples/storm-starter/src/jvm/storm/starter/spout/TwitterSampleSpout.java
./apache-storm-0.9.6/examples/storm-starter/src/jvm/storm/starter/tools/
./apache-storm-0.9.6/examples/storm-starter/src/jvm/storm/starter/tools/SlotBasedCounter.java
./apache-storm-0.9.6/examples/storm-starter/src/jvm/storm/starter/tools/Rankable.java
./apache-storm-0.9.6/examples/storm-starter/src/jvm/storm/starter/tools/SlidingWindowCounter.java
./apache-storm-0.9.6/examples/storm-starter/src/jvm/storm/starter/tools/RankableObjectWithFields.java
./apache-storm-0.9.6/examples/storm-starter/src/jvm/storm/starter/tools/Rankings.java
./apache-storm-0.9.6/examples/storm-starter/src/jvm/storm/starter/tools/NthLastModifiedTimeTracker.java
./apache-storm-0.9.6/examples/storm-starter/src/jvm/storm/starter/util/
./apache-storm-0.9.6/examples/storm-starter/src/jvm/storm/starter/util/StormRunner.java
./apache-storm-0.9.6/examples/storm-starter/src/jvm/storm/starter/util/TupleHelpers.java
./apache-storm-0.9.6/examples/storm-starter/src/jvm/storm/starter/ExclamationTopology.java
./apache-storm-0.9.6/examples/storm-starter/src/jvm/storm/starter/ReachTopology.java
./apache-storm-0.9.6/examples/storm-starter/src/jvm/storm/starter/ManualDRPC.java
./apache-storm-0.9.6/examples/storm-starter/src/jvm/storm/starter/bolt/
./apache-storm-0.9.6/examples/storm-starter/src/jvm/storm/starter/bolt/TotalRankingsBolt.java
./apache-storm-0.9.6/examples/storm-starter/src/jvm/storm/starter/bolt/AbstractRankerBolt.java
./apache-storm-0.9.6/examples/storm-starter/src/jvm/storm/starter/bolt/SingleJoinBolt.java
./apache-storm-0.9.6/examples/storm-starter/src/jvm/storm/starter/bolt/IntermediateRankingsBolt.java
./apache-storm-0.9.6/examples/storm-starter/src/jvm/storm/starter/bolt/PrinterBolt.java
./apache-storm-0.9.6/examples/storm-starter/src/jvm/storm/starter/bolt/RollingCountBolt.java
./apache-storm-0.9.6/examples/storm-starter/src/jvm/storm/starter/SingleJoinExample.java
./apache-storm-0.9.6/examples/storm-starter/src/jvm/storm/starter/trident/
./apache-storm-0.9.6/examples/storm-starter/src/jvm/storm/starter/trident/TridentWordCount.java
./apache-storm-0.9.6/examples/storm-starter/src/jvm/storm/starter/trident/TridentReach.java
./apache-storm-0.9.6/examples/storm-starter/src/jvm/storm/starter/WordCountTopologyNode.java
./apache-storm-0.9.6/examples/storm-starter/src/jvm/storm/starter/BasicDRPCTopology.java
./apache-storm-0.9.6/examples/storm-starter/src/jvm/storm/starter/PrintSampleStream.java
./apache-storm-0.9.6/examples/storm-starter/src/jvm/storm/starter/TransactionalWords.java
./apache-storm-0.9.6/examples/storm-starter/src/jvm/storm/starter/RollingTopWords.java
./apache-storm-0.9.6/examples/storm-starter/test/
./apache-storm-0.9.6/examples/storm-starter/test/jvm/
./apache-storm-0.9.6/examples/storm-starter/test/jvm/storm/
./apache-storm-0.9.6/examples/storm-starter/test/jvm/storm/starter/
./apache-storm-0.9.6/examples/storm-starter/test/jvm/storm/starter/tools/
./apache-storm-0.9.6/examples/storm-starter/test/jvm/storm/starter/tools/MockTupleHelpers.java
./apache-storm-0.9.6/examples/storm-starter/test/jvm/storm/starter/tools/NthLastModifiedTimeTrackerTest.java
./apache-storm-0.9.6/examples/storm-starter/test/jvm/storm/starter/tools/RankableObjectWithFieldsTest.java
./apache-storm-0.9.6/examples/storm-starter/test/jvm/storm/starter/tools/RankingsTest.java
./apache-storm-0.9.6/examples/storm-starter/test/jvm/storm/starter/tools/SlotBasedCounterTest.java
./apache-storm-0.9.6/examples/storm-starter/test/jvm/storm/starter/tools/SlidingWindowCounterTest.java
./apache-storm-0.9.6/examples/storm-starter/test/jvm/storm/starter/bolt/
./apache-storm-0.9.6/examples/storm-starter/test/jvm/storm/starter/bolt/TotalRankingsBoltTest.java
./apache-storm-0.9.6/examples/storm-starter/test/jvm/storm/starter/bolt/RollingCountBoltTest.java
./apache-storm-0.9.6/examples/storm-starter/test/jvm/storm/starter/bolt/IntermediateRankingsBoltTest.java
./apache-storm-0.9.6/examples/storm-starter/storm-starter-topologies-0.9.6.jar
./apache-storm-0.9.6/examples/storm-starter/multilang/
./apache-storm-0.9.6/examples/storm-starter/multilang/resources/
./apache-storm-0.9.6/examples/storm-starter/multilang/resources/asyncSplitsentence.js
./apache-storm-0.9.6/examples/storm-starter/multilang/resources/splitsentence.rb
./apache-storm-0.9.6/examples/storm-starter/multilang/resources/storm.rb
./apache-storm-0.9.6/examples/storm-starter/multilang/resources/splitsentence.js
./apache-storm-0.9.6/examples/storm-starter/multilang/resources/splitsentence.py
./apache-storm-0.9.6/examples/storm-starter/multilang/resources/storm.js
./apache-storm-0.9.6/examples/storm-starter/multilang/resources/storm.py
./apache-storm-0.9.6/examples/storm-starter/multilang/resources/randomsentence.js
./apache-storm-0.9.6/examples/storm-starter/README.markdown
./apache-storm-0.9.6/examples/storm-starter/pom.xml
./apache-storm-0.9.6/SECURITY.md
./apache-storm-0.9.6/external/
./apache-storm-0.9.6/external/storm-hdfs/
./apache-storm-0.9.6/external/storm-hdfs/storm-hdfs-0.9.6-sources.jar
./apache-storm-0.9.6/external/storm-hdfs/README.md
./apache-storm-0.9.6/external/storm-hdfs/storm-hdfs-0.9.6.jar
./apache-storm-0.9.6/external/storm-hdfs/storm-hdfs-0.9.6-javadoc.jar
./apache-storm-0.9.6/external/storm-kafka/
./apache-storm-0.9.6/external/storm-kafka/README.md
./apache-storm-0.9.6/external/storm-kafka/storm-kafka-0.9.6-sources.jar
./apache-storm-0.9.6/external/storm-kafka/storm-kafka-0.9.6-javadoc.jar
./apache-storm-0.9.6/external/storm-kafka/storm-kafka-0.9.6.jar
./apache-storm-0.9.6/external/storm-hbase/
./apache-storm-0.9.6/external/storm-hbase/storm-hbase-0.9.6-javadoc.jar
./apache-storm-0.9.6/external/storm-hbase/README.md
./apache-storm-0.9.6/external/storm-hbase/storm-hbase-0.9.6.jar
./apache-storm-0.9.6/external/storm-hbase/storm-hbase-0.9.6-sources.jar
./apache-storm-0.9.6/bin/
./apache-storm-0.9.6/bin/storm.cmd
./apache-storm-0.9.6/bin/storm
./apache-storm-0.9.6/bin/storm-config.cmd
./apache-storm-0.9.6/conf/
./apache-storm-0.9.6/conf/storm.yaml
./apache-storm-0.9.6/conf/storm_env.ini
./apache-storm-0.9.6/NOTICE
./apache-storm-0.9.6/public/
./apache-storm-0.9.6/public/topology.html
./apache-storm-0.9.6/public/css/
./apache-storm-0.9.6/public/css/style.css
./apache-storm-0.9.6/public/css/bootstrap-1.4.0.css
./apache-storm-0.9.6/public/component.html
./apache-storm-0.9.6/public/templates/
./apache-storm-0.9.6/public/templates/component-page-template.html
./apache-storm-0.9.6/public/templates/index-page-template.html
./apache-storm-0.9.6/public/templates/json-error-template.html
./apache-storm-0.9.6/public/templates/topology-page-template.html
./apache-storm-0.9.6/public/images/
./apache-storm-0.9.6/public/images/spinner.gif
./apache-storm-0.9.6/public/js/
./apache-storm-0.9.6/public/js/jquery-1.6.2.min.js
./apache-storm-0.9.6/public/js/moment.min.js
./apache-storm-0.9.6/public/js/bootstrap-twipsy.js
./apache-storm-0.9.6/public/js/jquery.tablesorter.min.js
./apache-storm-0.9.6/public/js/visualization.js
./apache-storm-0.9.6/public/js/jquery.mustache.js
./apache-storm-0.9.6/public/js/arbor-graphics.js
./apache-storm-0.9.6/public/js/arbor-tween.js
./apache-storm-0.9.6/public/js/jquery.blockUI.min.js
./apache-storm-0.9.6/public/js/script.js
./apache-storm-0.9.6/public/js/url.min.js
./apache-storm-0.9.6/public/js/jquery.cookies.2.2.0.min.js
./apache-storm-0.9.6/public/js/arbor.js
./apache-storm-0.9.6/public/index.html
./apache-storm-0.9.6/LICENSE
./apache-storm-0.9.6/lib/
./apache-storm-0.9.6/lib/clj-time-0.4.1.jar
./apache-storm-0.9.6/lib/tools.cli-0.2.4.jar
./apache-storm-0.9.6/lib/commons-lang-2.5.jar
./apache-storm-0.9.6/lib/tools.macro-0.1.0.jar
./apache-storm-0.9.6/lib/joda-time-2.0.jar
./apache-storm-0.9.6/lib/tools.logging-0.2.3.jar
./apache-storm-0.9.6/lib/objenesis-1.2.jar
./apache-storm-0.9.6/lib/commons-exec-1.1.jar
./apache-storm-0.9.6/lib/ring-core-1.1.5.jar
./apache-storm-0.9.6/lib/jetty-util-6.1.26.jar
./apache-storm-0.9.6/lib/chill-java-0.3.5.jar
./apache-storm-0.9.6/lib/json-simple-1.1.jar
./apache-storm-0.9.6/lib/jgrapht-core-0.9.0.jar
./apache-storm-0.9.6/lib/log4j-over-slf4j-1.6.6.jar
./apache-storm-0.9.6/lib/logback-classic-1.0.13.jar
./apache-storm-0.9.6/lib/ring-devel-0.3.11.jar
./apache-storm-0.9.6/lib/logback-core-1.0.13.jar
./apache-storm-0.9.6/lib/commons-logging-1.1.3.jar
./apache-storm-0.9.6/lib/disruptor-2.10.4.jar
./apache-storm-0.9.6/lib/compojure-1.1.3.jar
./apache-storm-0.9.6/lib/asm-4.0.jar
./apache-storm-0.9.6/lib/math.numeric-tower-0.0.1.jar
./apache-storm-0.9.6/lib/commons-io-2.4.jar
./apache-storm-0.9.6/lib/carbonite-1.4.0.jar
./apache-storm-0.9.6/lib/core.incubator-0.1.0.jar
./apache-storm-0.9.6/lib/reflectasm-1.07-shaded.jar
./apache-storm-0.9.6/lib/snakeyaml-1.11.jar
./apache-storm-0.9.6/lib/commons-codec-1.6.jar
./apache-storm-0.9.6/lib/minlog-1.2.jar
./apache-storm-0.9.6/lib/commons-fileupload-1.2.1.jar
./apache-storm-0.9.6/lib/hiccup-0.3.6.jar
./apache-storm-0.9.6/lib/ring-servlet-0.3.11.jar
./apache-storm-0.9.6/lib/clout-1.0.1.jar
./apache-storm-0.9.6/lib/ring-jetty-adapter-0.3.11.jar
./apache-storm-0.9.6/lib/servlet-api-2.5.jar
./apache-storm-0.9.6/lib/storm-core-0.9.6.jar
./apache-storm-0.9.6/lib/jetty-6.1.26.jar
./apache-storm-0.9.6/lib/jline-2.11.jar
./apache-storm-0.9.6/lib/slf4j-api-1.7.5.jar
./apache-storm-0.9.6/lib/kryo-2.21.jar
./apache-storm-0.9.6/lib/clj-stacktrace-0.2.2.jar
./apache-storm-0.9.6/lib/clojure-1.5.1.jar
./apache-storm-0.9.6/DISCLAIMER
./apache-storm-0.9.6/RELEASE
./apache-storm-0.9.6/logback/
./apache-storm-0.9.6/logback/cluster.xml
./apache-storm-0.9.6/README.markdown
 
Colored by Color Scripter
cs

8. storm 압축 파일을 클러스터링 된 각 노드에 배포합니다.

1
2
3
$ scp ./apache-storm-0.9.6.tar.gz hadoop-user@secondary:~/
apache-storm-0.9.6.tar.gz                                                          100%   20MB  20.0MB/s   00:00
 
Colored by Color Scripter
cs

9. 각 노드에서 2~6번까지의 작업을 진행합니다.

10. storm의 홈 디렉터리 이하 bin 디렉터리에서 실행합니다.

10-1. master node

: nimbus

1
2
3
4
$ ./bin/storm nimbus &
[1] 5741
Running: /usr/lib/jvm/java-8-oracle/bin/java -server -Dstorm.options= -Dstorm.home=/usr/local/apache-storm-0.9.6 -Dstorm.log.dir=/usr/local/apache-storm-0.9.6/logs -Djava.library.path=/usr/local/lib:/opt/local/lib:/usr/lib -Dstorm.conf.file= -cp /usr/local/apache-storm-0.9.6/lib/clj-time-0.4.1.jar:/usr/local/apache-storm-0.9.6/lib/tools.cli-0.2.4.jar:/usr/local/apache-storm-0.9.6/lib/commons-lang-2.5.jar:/usr/local/apache-storm-0.9.6/lib/tools.macro-0.1.0.jar:/usr/local/apache-storm-0.9.6/lib/joda-time-2.0.jar:/usr/local/apache-storm-0.9.6/lib/tools.logging-0.2.3.jar:/usr/local/apache-storm-0.9.6/lib/objenesis-1.2.jar:/usr/local/apache-storm-0.9.6/lib/commons-exec-1.1.jar:/usr/local/apache-storm-0.9.6/lib/ring-core-1.1.5.jar:/usr/local/apache-storm-0.9.6/lib/jetty-util-6.1.26.jar:/usr/local/apache-storm-0.9.6/lib/chill-java-0.3.5.jar:/usr/local/apache-storm-0.9.6/lib/json-simple-1.1.jar:/usr/local/apache-storm-0.9.6/lib/jgrapht-core-0.9.0.jar:/usr/local/apache-storm-0.9.6/lib/log4j-over-slf4j-1.6.6.jar:/usr/local/apache-storm-0.9.6/lib/logback-classic-1.0.13.jar:/usr/local/apache-storm-0.9.6/lib/ring-devel-0.3.11.jar:/usr/local/apache-storm-0.9.6/lib/logback-core-1.0.13.jar:/usr/local/apache-storm-0.9.6/lib/commons-logging-1.1.3.jar:/usr/local/apache-storm-0.9.6/lib/disruptor-2.10.4.jar:/usr/local/apache-storm-0.9.6/lib/compojure-1.1.3.jar:/usr/local/apache-storm-0.9.6/lib/asm-4.0.jar:/usr/local/apache-storm-0.9.6/lib/math.numeric-tower-0.0.1.jar:/usr/local/apache-storm-0.9.6/lib/commons-io-2.4.jar:/usr/local/apache-storm-0.9.6/lib/carbonite-1.4.0.jar:/usr/local/apache-storm-0.9.6/lib/core.incubator-0.1.0.jar:/usr/local/apache-storm-0.9.6/lib/reflectasm-1.07-shaded.jar:/usr/local/apache-storm-0.9.6/lib/snakeyaml-1.11.jar:/usr/local/apache-storm-0.9.6/lib/commons-codec-1.6.jar:/usr/local/apache-storm-0.9.6/lib/minlog-1.2.jar:/usr/local/apache-storm-0.9.6/lib/commons-fileupload-1.2.1.jar:/usr/local/apache-storm-0.9.6/lib/hiccup-0.3.6.jar:/usr/local/apache-storm-0.9.6/lib/ring-servlet-0.3.11.jar:/usr/local/apache-storm-0.9.6/lib/clout-1.0.1.jar:/usr/local/apache-storm-0.9.6/lib/ring-jetty-adapter-0.3.11.jar:/usr/local/apache-storm-0.9.6/lib/servlet-api-2.5.jar:/usr/local/apache-storm-0.9.6/lib/storm-core-0.9.6.jar:/usr/local/apache-storm-0.9.6/lib/jetty-6.1.26.jar:/usr/local/apache-storm-0.9.6/lib/jline-2.11.jar:/usr/local/apache-storm-0.9.6/lib/slf4j-api-1.7.5.jar:/usr/local/apache-storm-0.9.6/lib/kryo-2.21.jar:/usr/local/apache-storm-0.9.6/lib/clj-stacktrace-0.2.2.jar:/usr/local/apache-storm-0.9.6/lib/clojure-1.5.1.jar:/usr/local/apache-storm-0.9.6/conf -Xmx1024m -Dlogfile.name=nimbus.log -Dlogback.configurationFile=/usr/local/apache-storm-0.9.6/logback/cluster.xml backtype.storm.daemon.nimbus
 
Colored by Color Scripter
cs

* ./bin/storm nimbus >> nimbus.log 2>&1 &

10-2. worker node

: supervisor

1
2
3
$ ./bin/storm supervisor
Running: /usr/lib/jvm/java-8-oracle/bin/java -server -Dstorm.options= -Dstorm.home=/usr/local/apache-storm-0.9.6 -Dstorm.log.dir=/usr/local/apache-storm-0.9.6/logs -Djava.library.path=/usr/local/lib:/opt/local/lib:/usr/lib -Dstorm.conf.file= -cp /usr/local/apache-storm-0.9.6/lib/math.numeric-tower-0.0.1.jar:/usr/local/apache-storm-0.9.6/lib/log4j-over-slf4j-1.6.6.jar:/usr/local/apache-storm-0.9.6/lib/commons-fileupload-1.2.1.jar:/usr/local/apache-storm-0.9.6/lib/tools.macro-0.1.0.jar:/usr/local/apache-storm-0.9.6/lib/minlog-1.2.jar:/usr/local/apache-storm-0.9.6/lib/json-simple-1.1.jar:/usr/local/apache-storm-0.9.6/lib/commons-io-2.4.jar:/usr/local/apache-storm-0.9.6/lib/ring-core-1.1.5.jar:/usr/local/apache-storm-0.9.6/lib/tools.logging-0.2.3.jar:/usr/local/apache-storm-0.9.6/lib/clj-time-0.4.1.jar:/usr/local/apache-storm-0.9.6/lib/logback-classic-1.0.13.jar:/usr/local/apache-storm-0.9.6/lib/clojure-1.5.1.jar:/usr/local/apache-storm-0.9.6/lib/chill-java-0.3.5.jar:/usr/local/apache-storm-0.9.6/lib/commons-lang-2.5.jar:/usr/local/apache-storm-0.9.6/lib/slf4j-api-1.7.5.jar:/usr/local/apache-storm-0.9.6/lib/asm-4.0.jar:/usr/local/apache-storm-0.9.6/lib/ring-devel-0.3.11.jar:/usr/local/apache-storm-0.9.6/lib/clout-1.0.1.jar:/usr/local/apache-storm-0.9.6/lib/ring-jetty-adapter-0.3.11.jar:/usr/local/apache-storm-0.9.6/lib/storm-core-0.9.6.jar:/usr/local/apache-storm-0.9.6/lib/jgrapht-core-0.9.0.jar:/usr/local/apache-storm-0.9.6/lib/tools.cli-0.2.4.jar:/usr/local/apache-storm-0.9.6/lib/hiccup-0.3.6.jar:/usr/local/apache-storm-0.9.6/lib/snakeyaml-1.11.jar:/usr/local/apache-storm-0.9.6/lib/commons-logging-1.1.3.jar:/usr/local/apache-storm-0.9.6/lib/disruptor-2.10.4.jar:/usr/local/apache-storm-0.9.6/lib/compojure-1.1.3.jar:/usr/local/apache-storm-0.9.6/lib/jetty-6.1.26.jar:/usr/local/apache-storm-0.9.6/lib/jetty-util-6.1.26.jar:/usr/local/apache-storm-0.9.6/lib/carbonite-1.4.0.jar:/usr/local/apache-storm-0.9.6/lib/core.incubator-0.1.0.jar:/usr/local/apache-storm-0.9.6/lib/servlet-api-2.5.jar:/usr/local/apache-storm-0.9.6/lib/commons-exec-1.1.jar:/usr/local/apache-storm-0.9.6/lib/clj-stacktrace-0.2.2.jar:/usr/local/apache-storm-0.9.6/lib/ring-servlet-0.3.11.jar:/usr/local/apache-storm-0.9.6/lib/logback-core-1.0.13.jar:/usr/local/apache-storm-0.9.6/lib/commons-codec-1.6.jar:/usr/local/apache-storm-0.9.6/lib/kryo-2.21.jar:/usr/local/apache-storm-0.9.6/lib/reflectasm-1.07-shaded.jar:/usr/local/apache-storm-0.9.6/lib/objenesis-1.2.jar:/usr/local/apache-storm-0.9.6/lib/jline-2.11.jar:/usr/local/apache-storm-0.9.6/lib/joda-time-2.0.jar:/usr/local/apache-storm-0.9.6/conf -Xmx256m -Dlogfile.name=supervisor.log -Dlogback.configurationFile=/usr/local/apache-storm-0.9.6/logback/cluster.xml backtype.storm.daemon.supervisor
 
Colored by Color Scripter
cs

* ./bin/storm supervisor >> supervisor.log 2>&1 &

10-3. all node

: logviewer

1
2
3
4
$ ./bin/storm logviewer &
[2] 24163
Running: /usr/lib/jvm/java-8-oracle/bin/java -server -Dstorm.options= -Dstorm.home=/usr/local/apache-storm-0.9.6 -Dstorm.log.dir=/usr/local/apache-storm-0.9.6/logs -Djava.library.path=/usr/local/lib:/opt/local/lib:/usr/lib -Dstorm.conf.file= -cp /usr/local/apache-storm-0.9.6/lib/clj-time-0.4.1.jar:/usr/local/apache-storm-0.9.6/lib/tools.cli-0.2.4.jar:/usr/local/apache-storm-0.9.6/lib/commons-lang-2.5.jar:/usr/local/apache-storm-0.9.6/lib/tools.macro-0.1.0.jar:/usr/local/apache-storm-0.9.6/lib/joda-time-2.0.jar:/usr/local/apache-storm-0.9.6/lib/tools.logging-0.2.3.jar:/usr/local/apache-storm-0.9.6/lib/objenesis-1.2.jar:/usr/local/apache-storm-0.9.6/lib/commons-exec-1.1.jar:/usr/local/apache-storm-0.9.6/lib/ring-core-1.1.5.jar:/usr/local/apache-storm-0.9.6/lib/jetty-util-6.1.26.jar:/usr/local/apache-storm-0.9.6/lib/chill-java-0.3.5.jar:/usr/local/apache-storm-0.9.6/lib/json-simple-1.1.jar:/usr/local/apache-storm-0.9.6/lib/jgrapht-core-0.9.0.jar:/usr/local/apache-storm-0.9.6/lib/log4j-over-slf4j-1.6.6.jar:/usr/local/apache-storm-0.9.6/lib/logback-classic-1.0.13.jar:/usr/local/apache-storm-0.9.6/lib/ring-devel-0.3.11.jar:/usr/local/apache-storm-0.9.6/lib/logback-core-1.0.13.jar:/usr/local/apache-storm-0.9.6/lib/commons-logging-1.1.3.jar:/usr/local/apache-storm-0.9.6/lib/disruptor-2.10.4.jar:/usr/local/apache-storm-0.9.6/lib/compojure-1.1.3.jar:/usr/local/apache-storm-0.9.6/lib/asm-4.0.jar:/usr/local/apache-storm-0.9.6/lib/math.numeric-tower-0.0.1.jar:/usr/local/apache-storm-0.9.6/lib/commons-io-2.4.jar:/usr/local/apache-storm-0.9.6/lib/carbonite-1.4.0.jar:/usr/local/apache-storm-0.9.6/lib/core.incubator-0.1.0.jar:/usr/local/apache-storm-0.9.6/lib/reflectasm-1.07-shaded.jar:/usr/local/apache-storm-0.9.6/lib/snakeyaml-1.11.jar:/usr/local/apache-storm-0.9.6/lib/commons-codec-1.6.jar:/usr/local/apache-storm-0.9.6/lib/minlog-1.2.jar:/usr/local/apache-storm-0.9.6/lib/commons-fileupload-1.2.1.jar:/usr/local/apache-storm-0.9.6/lib/hiccup-0.3.6.jar:/usr/local/apache-storm-0.9.6/lib/ring-servlet-0.3.11.jar:/usr/local/apache-storm-0.9.6/lib/clout-1.0.1.jar:/usr/local/apache-storm-0.9.6/lib/ring-jetty-adapter-0.3.11.jar:/usr/local/apache-storm-0.9.6/lib/servlet-api-2.5.jar:/usr/local/apache-storm-0.9.6/lib/storm-core-0.9.6.jar:/usr/local/apache-storm-0.9.6/lib/jetty-6.1.26.jar:/usr/local/apache-storm-0.9.6/lib/jline-2.11.jar:/usr/local/apache-storm-0.9.6/lib/slf4j-api-1.7.5.jar:/usr/local/apache-storm-0.9.6/lib/kryo-2.21.jar:/usr/local/apache-storm-0.9.6/lib/clj-stacktrace-0.2.2.jar:/usr/local/apache-storm-0.9.6/lib/clojure-1.5.1.jar:/usr/local/apache-storm-0.9.6:/usr/local/apache-storm-0.9.6/conf -Xmx128m -Dlogfile.name=logviewer.log -Dlogback.configurationFile=/usr/local/apache-storm-0.9.6/logback/cluster.xml backtype.storm.daemon.logviewer
 
Colored by Color Scripter
cs

* ./bin/storm logviewer >> logviewer.log 2>&1 &

11. Example Topology를 실행합니다. 단, master node에서만 실행합니다.

1
2
3
4
5
6
7
8
9
10
$ ./bin/storm jar ./examples/storm-starter/storm-starter-topologies-0.9.6.jar storm.starter.RollingTopWords production-topology remote
Running: /usr/lib/jvm/java-8-oracle/bin/java -client -Dstorm.options= -Dstorm.home=/usr/local/apache-storm-0.9.6 -Dstorm.log.dir=/usr/local/apache-storm-0.9.6/logs -Djava.library.path=/usr/local/lib:/opt/local/lib:/usr/lib -Dstorm.conf.file= -cp /usr/local/apache-storm-0.9.6/lib/clj-time-0.4.1.jar:/usr/local/apache-storm-0.9.6/lib/tools.cli-0.2.4.jar:/usr/local/apache-storm-0.9.6/lib/commons-lang-2.5.jar:/usr/local/apache-storm-0.9.6/lib/tools.macro-0.1.0.jar:/usr/local/apache-storm-0.9.6/lib/joda-time-2.0.jar:/usr/local/apache-storm-0.9.6/lib/tools.logging-0.2.3.jar:/usr/local/apache-storm-0.9.6/lib/objenesis-1.2.jar:/usr/local/apache-storm-0.9.6/lib/commons-exec-1.1.jar:/usr/local/apache-storm-0.9.6/lib/ring-core-1.1.5.jar:/usr/local/apache-storm-0.9.6/lib/jetty-util-6.1.26.jar:/usr/local/apache-storm-0.9.6/lib/chill-java-0.3.5.jar:/usr/local/apache-storm-0.9.6/lib/json-simple-1.1.jar:/usr/local/apache-storm-0.9.6/lib/jgrapht-core-0.9.0.jar:/usr/local/apache-storm-0.9.6/lib/log4j-over-slf4j-1.6.6.jar:/usr/local/apache-storm-0.9.6/lib/logback-classic-1.0.13.jar:/usr/local/apache-storm-0.9.6/lib/ring-devel-0.3.11.jar:/usr/local/apache-storm-0.9.6/lib/logback-core-1.0.13.jar:/usr/local/apache-storm-0.9.6/lib/commons-logging-1.1.3.jar:/usr/local/apache-storm-0.9.6/lib/disruptor-2.10.4.jar:/usr/local/apache-storm-0.9.6/lib/compojure-1.1.3.jar:/usr/local/apache-storm-0.9.6/lib/asm-4.0.jar:/usr/local/apache-storm-0.9.6/lib/math.numeric-tower-0.0.1.jar:/usr/local/apache-storm-0.9.6/lib/commons-io-2.4.jar:/usr/local/apache-storm-0.9.6/lib/carbonite-1.4.0.jar:/usr/local/apache-storm-0.9.6/lib/core.incubator-0.1.0.jar:/usr/local/apache-storm-0.9.6/lib/reflectasm-1.07-shaded.jar:/usr/local/apache-storm-0.9.6/lib/snakeyaml-1.11.jar:/usr/local/apache-storm-0.9.6/lib/commons-codec-1.6.jar:/usr/local/apache-storm-0.9.6/lib/minlog-1.2.jar:/usr/local/apache-storm-0.9.6/lib/commons-fileupload-1.2.1.jar:/usr/local/apache-storm-0.9.6/lib/hiccup-0.3.6.jar:/usr/local/apache-storm-0.9.6/lib/ring-servlet-0.3.11.jar:/usr/local/apache-storm-0.9.6/lib/clout-1.0.1.jar:/usr/local/apache-storm-0.9.6/lib/ring-jetty-adapter-0.3.11.jar:/usr/local/apache-storm-0.9.6/lib/servlet-api-2.5.jar:/usr/local/apache-storm-0.9.6/lib/storm-core-0.9.6.jar:/usr/local/apache-storm-0.9.6/lib/jetty-6.1.26.jar:/usr/local/apache-storm-0.9.6/lib/jline-2.11.jar:/usr/local/apache-storm-0.9.6/lib/slf4j-api-1.7.5.jar:/usr/local/apache-storm-0.9.6/lib/kryo-2.21.jar:/usr/local/apache-storm-0.9.6/lib/clj-stacktrace-0.2.2.jar:/usr/local/apache-storm-0.9.6/lib/clojure-1.5.1.jar:./examples/storm-starter/storm-starter-topologies-0.9.6.jar:/usr/local/apache-storm-0.9.6/conf:/usr/local/apache-storm-0.9.6/bin -Dstorm.jar=./examples/storm-starter/storm-starter-topologies-0.9.6.jar storm.starter.RollingTopWords production-topology remote
209  [main] INFO  storm.starter.RollingTopWords - Topology name: production-topology
250  [main] INFO  storm.starter.RollingTopWords - Running in remote (cluster) mode
414  [main] INFO  backtype.storm.StormSubmitter - Jar not uploaded to master yet. Submitting jar...
426  [main] INFO  backtype.storm.StormSubmitter - Uploading topology jar ./examples/storm-starter/storm-starter-topologies-0.9.6.jar to assigned location: storm-local/nimbus/inbox/stormjar-887a116a-6fab-4846-8b61-84bba78d934f.jar
460  [main] INFO  backtype.storm.StormSubmitter - Successfully uploaded topology jar to assigned location: storm-local/nimbus/inbox/stormjar-887a116a-6fab-4846-8b61-84bba78d934f.jar
460  [main] INFO  backtype.storm.StormSubmitter - Submitting topology production-topology in distributed mode with conf {"topology.debug":true}
752  [main] INFO  backtype.storm.StormSubmitter - Finished submitting topology: production-topology
 
Colored by Color Scripter
cs

저작자표시

'Big Data Platform > Storm' 카테고리의 다른 글

[에러] Address already in use (Bind failed) (0)	2018.03.21

PREV 1 NEXT

Another Windows