Hive 다운로드
wget archive.apache.org/dist/hive/hive-3.1.2/apache-hive-3.1.2-bin.tar.gz
$ wget archive.apache.org/dist/hive/hive-3.1.2/apache-hive-3.1.2-bin.tar.gz
$ tar -zxvf apache-hive-3.1.2-bin.tar.gz
$ cp -r apache-hive-3.1.2-bin /usr/local/hive
Hive 환경변수 설정
$ cat ~/.bashrc | grep HIVE
export HIVE_HOME=/usr/local/hive
export HIVE_CONF_DIR=/usr/local/hive/conf
export PATH=$PATH:$JAVA_HOME/bin:$HADOOP_HOME/sbin:$SPARK_HOME/bin:$SQOOP_HOME/bin:$HIVE_HOME/bin
Hive 설정
//hive디렉토리 권한 설정
$ chown -R hadoopuser:hadoopuser hive
//hive-env.sh 설정
$ cd $HIVE_HOME/conf
$ cp hive-env.sh.template hive-env.sh
$ cat hive-env.sh | grep HADOOP_HOME
# Set HADOOP_HOME to point to a specific hadoop install directory
HADOOP_HOME=/usr/local/hadoop
//hive-site.xml 설정(hive-site.xml 파일 생성)
$ cat hive-site.xml
<?xml version="1.0" encoding="UTF-8" standalone="no"?>
<?xml-stylesheet type="text/xsl" href="configuration.xsl"?>
<configuration>
<property>
<name>hive.metastore.local</name>
<value>false</value>
</property>
<property>
<name>hive.metastore.warehouse.dir</name>
<value>hdfs://하둡마스터호스트:9000/user/hive</value>
</property>
Hive를 위한 HDFS 디렉토리 생성
$ hadoop fs -mkdir /user/hive
$ hadoop fs -ls /user/
Found 3 items
drwxr-xr-x - hadoopuser supergroup 0 2020-12-09 22:44 /user/hadoopuser
drwxr-xr-x - hadoopuser supergroup 0 2020-12-13 23:57 /user/hive
drwxr-xr-x - hadoopuser supergroup 0 2020-12-09 23:03 /user/songhee
Hive실행
메타스토어 초기화 중 에러 발생
이 경우, hive/lib/guava-19.0.jar를 삭제하고, hadoop lib중 guava-27.0-jre.jar를 hive/lib로 복사한다.
//메타스토어 초기화
$ schematool -dbType derby -initSchema
//에러
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/usr/local/hadoop/share/hadoop/common/lib/slf4j-log4j12-1.7.25.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/usr/local/hive/lib/log4j-slf4j-impl-2.10.0.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]
2020-12-14 00:02:07,226 INFO [main] conf.HiveConf (HiveConf.java:findConfigFile(187)) - Found configuration file file:/usr/local/hive/conf/hive-site.xml
Exception in thread "main" java.lang.NoSuchMethodError: com.google.common.base.Preconditions.checkArgument(ZLjava/lang/String;Ljava/lang/Object;)V
at org.apache.hadoop.conf.Configuration.set(Configuration.java:1357)
at org.apache.hadoop.conf.Configuration.set(Configuration.java:1338)
at org.apache.hadoop.mapred.JobConf.setJar(JobConf.java:536)
at org.apache.hadoop.mapred.JobConf.setJarByClass(JobConf.java:554)
at org.apache.hadoop.mapred.JobConf.<init>(JobConf.java:448)
at org.apache.hadoop.hive.conf.HiveConf.initialize(HiveConf.java:5141)
at org.apache.hadoop.hive.conf.HiveConf.<init>(HiveConf.java:5104)
at org.apache.hive.beeline.HiveSchemaTool.<init>(HiveSchemaTool.java:96)
at org.apache.hive.beeline.HiveSchemaTool.main(HiveSchemaTool.java:1473)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.apache.hadoop.util.RunJar.run(RunJar.java:323)
at org.apache.hadoop.util.RunJar.main(RunJar.java:236)
$ rm $HIVE_HOME/lib/guava-19.0.jar
$ cp /usr/local/hadoop/share/hadoop/hdfs/lib/guava-27.0-jre.jar /usr/local/hive/lib/
재시도
$ cp $HIVE_HOME/bin
$ ./schematool -dbType derby -initSchema --verbose
schemaTool completed
$ hive
설치 완료!
Hive 기본 명령어
$ hive
//DB생성
hive> create database test;
hive> show databases;
//테이블 생성
hive> create table test.tbl (
> col1 integer,
> col2 string);
//insert
hive> insert into table test.tbl
> select 1 as col1, 'song' as col2;
//select
hive > select * from test.tbl;
HDFS 확인
$ hadoop fs -ls /user/hive/warehouse/test.db/tbl
Found 1 items
-rw-r--r-- 2 hadoopuser supergroup 7 2020-12-15 02:16 /user/hive/warehouse/test.db/tbl/000000_0
$ hadoop fs -cat /user/hive/warehouse/test.db/tbl/000000_0
1song
'Hive' 카테고리의 다른 글
Hive Partition 추가 및 변경 (ALTER TABLE, MSCK REPAIR) (0) | 2024.04.18 |
---|---|
hive 설정 및 실행 - hive-site.xml, hiveserver 다른 포트로 띄우기 (0) | 2024.04.18 |
Sqoop을 이용하여 MySQL의 데이터를 HIVE로 조회해보자! (0) | 2020.12.15 |