CM部署CDH安装组件到Hive时报错: Cause:Table 'scm.VERSION' doesn't exist.
Cloudera Manager部署CDH到首次运行页面,安装运行Hive时,报错:HiveMetaException: Failed to get schema version, Cause:Table ‘metastore.VERSION’ doesn’t exist。
报错背景:
CM和CDH版本为5.15.2
①初始化CM数据库时执行的命令是:
./scm_prepare_database.sh mysql scm scm
即数据库类型是MySQL,scm数据库,scm数据库用户;
②在“首次运行”页面之前需要对Hive设置数据库,这里也对Hive所用数据库设置为scm数据库,即是Hive数据库和CM初始化时用的是同一个scm库,然后在安装运行Hive报该错误。
③进入MySQL的scm库,发现确实缺少VERSION表。
④通过查阅文章,感觉是Hive用的库和CM用的库冲突;
我是这样解决的:
停掉CM Server,drop掉scm库,在MySQL里重新新建hive库和scm库:
drop database scm;
CREATE DATABASE scm DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8_general_ci;
CREATE DATABASE hive DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8_general_ci;
新建完两个库之后,执行CM数据库的初始化脚本:
./opt/cloudera-manager/cm-5.15.1/share/cmf/schema/scm_prepare_database.sh mysql scm scm
然后在scm库中执行脚本:
source /opt/cloudera/parcels/CDH-5.15.2-1.cdh5.15.2.p0.3/lib/hive/scripts/metastore/upgrade/mysql/hive-schema-1.1.0.mysql.sql
此脚本是为了解决VERSION表,执行完后发现scm库中有VERSION表了。
再次把CM Server运行起来,CM Agent重启一下,重新回到CM设置组件所用数据库的页面,注意这里Hive需要设置为MySQL的hive库(与CM所用的scm库独立),用户均为scm,继续下一步安装运行,发现全绿!
查看一下此时hive库和scm库的表:
±-------------------------------+
| Tables_in_scm |
±-------------------------------+
| AUDITS |
| BUCKETING_COLS |
| BUNDLE_ACTIONS |
| BUNDLE_JOBS |
| CDS |
| CLIENT_CONFIGS |
| CLIENT_CONFIGS_TO_HOSTS |
| CLUSTERS |
| CLUSTERS_AUD |
| CLUSTER_ACTIVATED_RELEASES |
| CLUSTER_ACTIVATED_RELEASES_AUD |
| CLUSTER_MANAGED_RELEASES |
| CLUSTER_UNDISTRIBUTED_RELEASES |
| CMON_ACTIVITY |
| CMON_ACTIVITY_ATTRIBUTE |
| CMON_ACTIVITY_DP |
| CMON_ACTIVITY_DP_2020_05_12 |
| CMON_ACT_LMV |
| CMON_ACT_LMV_2020_04_29 |
| CMON_ACT_LMV_2020_04_30 |
| CMON_ACT_LMV_2020_05_01 |
| CMON_ACT_LMV_2020_05_02 |
| CMON_ACT_LMV_2020_05_03 |
| CMON_ACT_LMV_2020_05_04 |
| CMON_ACT_LMV_2020_05_05 |
| CMON_ACT_LMV_2020_05_06 |
| CMON_ACT_LMV_2020_05_07 |
| CMON_ACT_LMV_2020_05_08 |
| CMON_ACT_LMV_2020_05_09 |
| CMON_ACT_LMV_2020_05_10 |
| CMON_ACT_LMV_2020_05_11 |
| CMON_ACT_LMV_2020_05_12 |
| CMON_ACT_LMV_2020_05_13 |
| CMON_ATTEMPT |
| CMON_ATTEMPT_2020_04_29 |
| CMON_ATTEMPT_2020_04_30 |
| CMON_ATTEMPT_2020_05_01 |
| CMON_ATTEMPT_2020_05_02 |
| CMON_ATTEMPT_2020_05_03 |
| CMON_ATTEMPT_2020_05_04 |
| CMON_ATTEMPT_2020_05_05 |
| CMON_ATTEMPT_2020_05_06 |
| CMON_ATTEMPT_2020_05_07 |
| CMON_ATTEMPT_2020_05_08 |
| CMON_ATTEMPT_2020_05_09 |
| CMON_ATTEMPT_2020_05_10 |
| CMON_ATTEMPT_2020_05_11 |
| CMON_ATTEMPT_2020_05_12 |
| CMON_ATTEMPT_2020_05_13 |
| CMON_ATT_LMV |
| CMON_ATT_LMV_2020_04_29 |
| CMON_ATT_LMV_2020_04_30 |
| CMON_ATT_LMV_2020_05_01 |
| CMON_ATT_LMV_2020_05_02 |
| CMON_ATT_LMV_2020_05_03 |
| CMON_ATT_LMV_2020_05_04 |
| CMON_ATT_LMV_2020_05_05 |
| CMON_ATT_LMV_2020_05_06 |
| CMON_ATT_LMV_2020_05_07 |
| CMON_ATT_LMV_2020_05_08 |
| CMON_ATT_LMV_2020_05_09 |
| CMON_ATT_LMV_2020_05_10 |
| CMON_ATT_LMV_2020_05_11 |
| CMON_ATT_LMV_2020_05_12 |
| CMON_ATT_LMV_2020_05_13 |
| CMON_LL_DP |
| CMON_LL_DP_2020_05_12 |
| CMON_LL_DP_HR |
| CMON_LL_DP_HR_2020_05_12 |
| CMON_LL_TIME_SERIES_ATTRIBUTE |
| CMON_LL_TIME_SERIES_ID |
| CMON_METRIC_INFO |
| CMON_SCHEMA_VERSION |
| CM_PEERS |
| CM_VERSION |
| COLUMNS_V2 |
| COMMANDS |
| COMMANDS_DETAIL |
| COMMAND_SCHEDULES |
| CONFIGS |
| CONFIGS_AUD |
| CONFIG_CONTAINERS |
| COORD_ACTIONS |
| COORD_JOBS |
| CREDENTIALS |
| DATABASE_PARAMS |
| DBS |
| DB_PRIVS |
| DELEGATION_TOKENS |
| DIAGNOSTICS_EVENTS |
| EXTERNAL_ACCOUNTS |
| EXTERNAL_ACCOUNTS_AUD |
| FUNCS |
| FUNC_RU |
| GLOBAL_PRIVS |
| GLOBAL_SETTINGS |
| HOSTS |
| HOSTS_AUD |
| HOST_TEMPLATES |
| HOST_TEMPLATE_TO_ROLE_CONF_GRP |
| IDXS |
| INDEX_PARAMS |
| MASTER_KEYS |
| METRICS |
| NOTIFICATION_LOG |
| NOTIFICATION_SEQUENCE |
| NUCLEUS_TABLES |
| OOZIE_SYS |
| OPENJPA_SEQUENCE_TABLE |
| PARCELS |
| PARCEL_COMPONENTS |
| PARTITIONS |
| PARTITION_EVENTS |
| PARTITION_INFO |
| PARTITION_KEYS |
| PARTITION_KEY_VALS |
| PARTITION_PARAMS |
| PART_COL_PRIVS |
| PART_COL_STATS |
| PART_PRIVS |
| PROCESSES |
| PROCESSES_DETAIL |
| PROCESS_ACTIVE_RELEASES |
| RELEASES |
| RELEASES_AUD |
| REVISIONS |
| ROLES |
| ROLES_AUD |
| ROLE_CONFIG_GROUPS |
| ROLE_CONFIG_GROUPS_AUD |
| ROLE_MAP |
| ROLE_STALENESS_STATUS |
| SCHEMA_VERSION |
| SDS |
| SD_PARAMS |
| SEQUENCE_TABLE |
| SERDES |
| SERDE_PARAMS |
| SERVICES |
| SERVICES_AUD |
| SKEWED_COL_NAMES |
| SKEWED_COL_VALUE_LOC_MAP |
| SKEWED_STRING_LIST |
| SKEWED_STRING_LIST_VALUES |
| SKEWED_VALUES |
| SLA_EVENTS |
| SLA_REGISTRATION |
| SLA_SUMMARY |
| SNAPSHOT_POLICIES |
| SORT_COLS |
| TABLE_PARAMS |
| TAB_COL_STATS |
| TBLS |
| TBL_COL_PRIVS |
| TBL_PRIVS |
| TYPES |
| TYPE_FIELDS |
| USERS |
| USER_ROLES |
| USER_SETTINGS |
| VALIDATE_CONN |
| VERSION |
| WF_ACTIONS |
| WF_JOBS |
| auth_group |
| auth_group_permissions |
| auth_permission |
| auth_user |
| auth_user_groups |
| auth_user_user_permissions |
| axes_accessattempt |
| axes_accesslog |
| beeswax_metainstall |
| beeswax_queryhistory |
| beeswax_savedquery |
| beeswax_session |
| defaultconfiguration_groups |
| desktop_defaultconfiguration |
| desktop_document |
| desktop_document2 |
| desktop_document2_dependencies |
| desktop_document2permission |
| desktop_document_tags |
| desktop_documentpermission |
| desktop_documenttag |
| desktop_settings |
| desktop_userpreferences |
| django_admin_log |
| django_content_type |
| django_openid_auth_association |
| django_openid_auth_nonce |
| django_openid_auth_useropenid |
| django_session |
| django_site |
| documentpermission2_groups |
| documentpermission2_users |
| documentpermission_groups |
| documentpermission_users |
| jobsub_checkforsetup |
| jobsub_jobdesign |
| jobsub_jobhistory |
| jobsub_oozieaction |
| jobsub_ooziedesign |
| jobsub_ooziejavaaction |
| jobsub_ooziemapreduceaction |
| jobsub_ooziestreamingaction |
| oozie_bundle |
| oozie_bundledcoordinator |
| oozie_coordinator |
| oozie_datainput |
| oozie_dataoutput |
| oozie_dataset |
| oozie_decision |
| oozie_decisionend |
| oozie_distcp |
| oozie_email |
| oozie_end |
| oozie_fork |
| oozie_fs |
| oozie_generic |
| oozie_history |
| oozie_hive |
| oozie_java |
| oozie_job |
| oozie_join |
| oozie_kill |
| oozie_link |
| oozie_mapreduce |
| oozie_node |
| oozie_pig |
| oozie_shell |
| oozie_sqoop |
| oozie_ssh |
| oozie_start |
| oozie_streaming |
| oozie_subworkflow |
| oozie_workflow |
| pig_document |
| pig_pigscript |
| search_collection |
| search_facet |
| search_result |
| search_sorting |
| south_migrationhistory |
| useradmin_grouppermission |
| useradmin_huepermission |
| useradmin_ldapgroup |
| useradmin_userprofile |
±-------------------------------+
±--------------------------+
| Tables_in_hive |
±--------------------------+
| BUCKETING_COLS |
| CDS |
| COLUMNS_V2 |
| COMPACTION_QUEUE |
| COMPLETED_TXN_COMPONENTS |
| DATABASE_PARAMS |
| DBS |
| DB_PRIVS |
| DELEGATION_TOKENS |
| FUNCS |
| FUNC_RU |
| GLOBAL_PRIVS |
| HIVE_LOCKS |
| IDXS |
| INDEX_PARAMS |
| MASTER_KEYS |
| METASTORE_DB_PROPERTIES |
| NEXT_COMPACTION_QUEUE_ID |
| NEXT_LOCK_ID |
| NEXT_TXN_ID |
| NOTIFICATION_LOG |
| NOTIFICATION_SEQUENCE |
| NUCLEUS_TABLES |
| PARTITIONS |
| PARTITION_EVENTS |
| PARTITION_KEYS |
| PARTITION_KEY_VALS |
| PARTITION_PARAMS |
| PART_COL_PRIVS |
| PART_COL_STATS |
| PART_PRIVS |
| ROLES |
| ROLE_MAP |
| SDS |
| SD_PARAMS |
| SEQUENCE_TABLE |
| SERDES |
| SERDE_PARAMS |
| SKEWED_COL_NAMES |
| SKEWED_COL_VALUE_LOC_MAP |
| SKEWED_STRING_LIST |
| SKEWED_STRING_LIST_VALUES |
| SKEWED_VALUES |
| SORT_COLS |
| TABLE_PARAMS |
| TAB_COL_STATS |
| TBLS |
| TBL_COL_PRIVS |
| TBL_PRIVS |
| TXNS |
| TXN_COMPONENTS |
| TYPES |
| TYPE_FIELDS |
| VERSION |
±--------------------------+
我解决该问题的主要操作:
①新建两个库:hive和scm,CM用scm库,Hive用hive库,不要混在同一个库一起用;
②初始化CM数据库之后,再执行如下脚本解决VERSION表不存在的问题:
scm库中执行脚本:
source /opt/cloudera/parcels/CDH-5.15.2-1.cdh5.15.2.p0.3/lib/hive/scripts/metastore/upgrade/mysql/hive-schema-1.1.0.mysql.sql。
最后,组件全绿启动之后,发现点击继续按钮还是停留在上面这个页面…
——————————————————————————————
本文为个人记录所用,如有侵权/错误,恳请指出 (╹▽╹)…