云计算之OpenStack实战记（一）与踩坑
一、了解Openstack
1.1 OpenStack简介
OpenStack是一整套开源软件项目的综合，它允许企业或服务提供者建立、运行自己的云计算和存储设施。Rackspace与NASA是最初重要的两个贡献者，前者提供了“云文件”平台代码，该平台增强了OpenStack对象存储部分的功能，而后者带来了“Nebula”平台形成了OpenStack其余的部分。而今，OpenStack基金会已经有150多个会员，包括很多知名公司如“Canonical、DELL、Citrix”等。
1.2 OpenStack的几大组件
DashBoard |
Horizon |
提供WEB界面 |
Computer |
Nova |
计算也就是虚拟机 |
Networking |
Neutron |
提供给nova网络支持 |
Object Storage |
Swift |
提供对象存储 |
Block Storage |
Cinder |
提供云硬盘给nova，同时备份到Swift |
Identity Sservice |
Keystone |
提供所有组件的认证 |
Image Service |
Glance |
提供给nova镜像服务 |
Telemetry Service |
Cellometer |
监控 cinder,neutron,nova,glance |
Orchestration Service |
Heat |
与AWS cloud兼容 |
1.2.1 图解各大组件之间关系
1.2.2 谈谈openstack的组件
OpenStack 认证(keystone)
Keystone为所有的OpenStack组件提供认证和访问策略服务，它依赖自身REST（基于Identity API）系统进行工作，主要对（但不限于）Swift、Glance、Nova等进行认证与授权。事实上，授权通过对动作消息来源者请求的合法性进行鉴定
Keystone采用两种授权方式，一种基于用户名/密码，另一种基于令牌（Token）。除此之外，Keystone提供以下三种服务：
a.令牌服务：含有授权用户的授权信息
b.目录服务：含有用户合法操作的可用服务列表
c.策略服务：利用Keystone具体指定用户或群组某些访问权限
认证服务组件
1）通过宾馆对比keystone
User 住宾馆的人
Credentials ×××
Authentication 认证你的×××
Token ×××
project 组间
Service 宾馆可以提供的服务类别，比如，饮食类，娱乐类
Endpoint 具体的一种服务，比如吃烧烤，打羽毛球
Role VIP 等级，VIP越高，享有越高的权限
2）keystone组件详细说明
a.服务入口endpoint：如Nova、Swift和Glance一样每个OpenStack服务都拥有一个指定的端口和专属的URL，我们称其为入口（endpoints）。
b.用户user：Keystone授权使用者
注：代表一个个体，OpenStack以用户的形式来授权服务给它们。用户拥有证书（credentials），且可能分配给一个或多个租户。经过验证后，会为每个单独的租户提供一个特定的令牌。
ｃ.服务service：总体而言，任何通过Keystone进行连接或管理的组件都被称为服务。举个例子，我们可以称Glance为Keystone的服务。
d.角色role：为了维护安全限定，就内特定用户可执行的操作而言，该用户关联的角色是非常重要的。注：一个角色是应是某个租户的使用权限集合，以允许某个指定用户访问或使用特定操作。角色是使用权限的逻辑分组，它使得通用的权限可以简单地分组并绑定到与某个指定租户相关的用户。
e.租间project：租间指的是具有全部服务入口并配有特定成员角色的一个项目。注：一个租间映射到一个Nova的“project-id”，在对象存储中，一个租间可以有多个容器。根据不同的安装方式，一个租间可以代表一个客户、帐号、组织或项目。
OpenStack Dashboard界面 (horizon)
Horizon是一个用以管理、控制OpenStack服务的Web控制面板，它可以管理实例、镜像、创建密匙对，对实例添加卷、操作Swift容器等。除此之外，用户还可以在控制面板中使用终端（console）或VNC直接访问实例。总之，Horizon具有如下一些特点：
a.实例管理：创建、终止实例，查看终端日志，VNC连接，添加卷等
b.访问与安全管理：创建安全群组，管理密匙对，设置浮动IP等
c.偏好设定：对虚拟硬件模板可以进行不同偏好设定
d.镜像管理：编辑或删除镜像
e.查看服务目录
f.管理用户、配额及项目用途
g.用户管理：创建用户等
h.卷管理：创建卷和快照
i.对象存储处理：创建、删除容器和对象
j.为项目下载环境变量
OpenStack nova
图解nova
API:负责接收和响应外部请求，支持OpenStackAPI，EC2API
nova-api 组件实现了RESTfulAPI功能，是外部访问Nova的唯一途径，接收外部的请求并通过Message Queue将请求发送给其他服务组件，同时也兼容EC2API，所以可以用EC2的管理工具对nova进行日常管理
Cert：负责身份认证
Scheduler：用于云主机调度
Nova Scheduler模块在openstack中的作用是决策虚拟机创建在哪个主机（计算节点），一般会根据过滤计算节点或者通过加权的方法调度计算节点来创建虚拟机。
1)过滤
首先得到未经过过滤的主机列表，然后根据过滤属性，选择服务条件的计算节点主机
2)调度
经过过滤后，需要对主机进行权值的计算，根据策略选择相应的某一台主机（对于每一个要创建的虚拟机而言）
注：Openstack默认不支持指定的计算节点创建虚拟机
你可以得到更多nova的知识==>>Nova过滤调度器
Conductor：计算节点访问，数据的中间件
Consloeauth：用于控制台的授权认证
Novncproxy：VNC代理
OpenStack 对象存储 (swift)
Swift为OpenStack提供一种分布式、持续虚拟对象存储，它类似于Amazon Web Service的S3简单存储服务。Swift具有跨节点百级对象的存储能力。Swift内建冗余和失效备援管理，也能够处理归档和媒体流，特别是对大数据（千兆字节）和大容量（多对象数量）的测度非常高效。
swift功能及特点
海量对象存储
大文件（对象）存储
数据冗余管理
归档能力—–处理大数据集
为虚拟机和云应用提供数据容器
处理流媒体
对象安全存储
备份与归档
良好的可伸缩性
Swift的组件
Swift账户
Swift容器
Swift对象
Swift代理
Swift RING
Swift代理服务器
用户都是通过Swift-API与代理服务器进行交互，代理服务器正是接收外界请求的门卫，它检测合法的实体位置并路由它们的请求。
此外，代理服务器也同时处理实体失效而转移时，故障切换的实体重复路由请求。
Swift对象服务器
对象服务器是一种二进制存储，它负责处理本地存储中的对象数据的存储、检索和删除。对象都是文件系统中存放的典型的二进制文件，具有扩展文件属性的元数据（xattr）。注：xattr格式被Linux中的ext3/4，XFS，Btrfs，JFS和ReiserFS所支持，但是并没有有效测试证明在XFS，JFS，ReiserFS，Reiser4和ZFS下也同样能运行良好。不过，XFS被认为是当前最好的选择。
Swift容器服务器
容器服务器将列出一个容器中的所有对象，默认对象列表将存储为SQLite文件（译者注：也可以修改为MySQL，安装中就是以MySQL为例）。容器服务器也会统计容器中包含的对象数量及容器的存储空间耗费。
Swift账户服务器
账户服务器与容器服务器类似，将列出容器中的对象。
Ring（索引环）
Ring容器记录着Swift中物理存储对象的位置信息，它是真实物理存储位置的实体名的虚拟映射，类似于查找及定位不同集群的实体真实物理位置的索引服务。这里所谓的实体指账户、容器、对象，它们都拥有属于自己的不同的Rings。
OpenStack 块存储(cinder)
API service：负责接受和处理Rest请求，并将请求放入RabbitMQ队列。Cinder提供VolumeAPI V2
Scheduler service:响应请求，读取或写向块存储数据库为维护状态，通过消息队列机制与其他进程交互，或直接与上层块存储提供的硬件或软件交互，通过driver结构，他可以与中队的存储
提供者进行交互
Volume service: 该服务运行在存储节点上，管理存储空间。每个存储节点都有一个Volume Service，若干个这样的存储节点联合起来可以构成一个存储资源池。为了支持不同类型和型号的存储
OpenStack Image service (glance)
glance 主要有三个部分构成：glance-api，glance-registry以及image store
glance-api：接受云系统镜像的创建，删除，读取请求
glance-registry：云系统的镜像注册服务
OpenStack 网络 (neutron)
这里就不详细介绍了，后面会有详细的讲解
二、环境准备
2.1 准备机器
本次实验使用的是VMvare虚拟机。详情如下
控制节点
hostname：node1.controller
外网网卡ip:eth0(192.168.8.150)
内网网卡ip:eth1(172.16.8.150)
系统及硬件：CentOS 7.2 内存4G，硬盘50G
计算节点：
hostname：node2.compute
外网网卡ip:eth0(192.168.8.155)
内网网卡ip:eth1(172.16.8.155)
系统及硬件：CentOS 7.2 内存2G，硬盘50G
2.2 OpenStack版本介绍
本文使用的是L（Liberty）版，其他版本如下图 还有M版。
2.3 安装组件服务
2.3.1 控制节点安装
Base
1 cp /etc/sysconfig/grub /etc/sysconfig/grub.ori
2 vi /etc/sysconfig/grub
3 在GRUB_CMDLINE_LINUX内容的最后面增加“net.ifnames=0 biosdevname=0”
4 diff /etc/sysconfig/grub.ori /etc/sysconfig/grub
6c6
< GRUB_CMDLINE_LINUX="crashkernel=auto rhgb quiet"
---
> GRUB_CMDLINE_LINUX="crashkernel=auto rhgb quiet net.ifnames=0 biosdevname=0"
5 grub2-mkconfig -o /boot/grub2/grub.cfg
6 init 6(重启系统后查看网卡名称)
7 vi /etc/hosts
127.0.0.1 localhost localhost.localdomain localhost4 localhost4.localdomain4
::1 localhost localhost.localdomain localhost6 localhost6.localdomain6
172.16.8.150 node1 node1.controller
172.16.8.155 node2 node2.compute
192.168.8.150 node1 node1.controller
192.168.8.155 node2 node2.compute
8 setenforce 0
9 systemctl stop firewalld
10 systemctl disable firewalld
11 ip ro li //查看路由
12 yum install http://dl.fedoraproject.org/pub/epel/7/x86_64/e/epel-release-7-5.noarch.rpm -y
13 yum install centos-release-openstack-liberty -y
14 yum install python-openstackclient -y
MySQL
15 yum install mariadb mariadb-server MySQL-python -y
RabbitMQ
16 yum install rabbitmq-server -y
Keystone
17 yum install openstack-keystone httpd mod_wsgi memcached python-memcached -y
Glance
18 yum install openstack-glance python-glance python-glanceclient -y
Nova
19 yum install openstack-nova-api openstack-nova-cert openstack-nova-conductor openstack-nova-console openstack-nova-novncproxy openstack-nova-scheduler python-novaclient -y
Neutron
20 yum install openstack-neutron openstack-neutron-ml2 openstack-neutron-linuxbridge python-neutronclient ebtables ipset -y
Dashboard
21 yum install openstack-dashboard -y
2.3.2 计算节点安装
Base
22 yum install http://dl.fedoraproject.org/pub/epel/7/x86_64/e/epel-release-7-5.noarch.rpm -y
23 yum install centos-release-openstack-liberty -y
24 yum install python-openstackclient -y
Nova
25 yum install openstack-nova-compute sysfsutils -y
Neutron
26 yum install openstack-neutron openstack-neutron-linuxbridge ebtables ipset -y
三、实战OpenStack之控制节点
3.1 CentOS7的时间同步服务器chrony
1 [[email protected] ~]# yum -y install chrony
修改其配置文件
2 [[email protected] ~]# vi /etc/chrony.conf
3 allow 192.168.0.0/24
chrony开机自启动，并且启动
4 [[email protected] ~]# systemctl enable chronyd.service
5 [[email protected] ~]# systemctl start chronyd.service
设置Centos7的时区
6 [[email protected] ~]# timedatectl set-timezone Asia/Shanghai
查看时区和时间
7 [[email protected] ~]# timedatectl status
3.2 入手mysql
Openstack的所有组件除了Horizon，都要用到数据库，本文使用的是mysql，在CentOS7中，默认叫做MariaDB。
拷贝配置文件
[[email protected] ~]# cp /usr/share/mariadb/my-medium.cnf /etc/my.cnf
cp: overwrite ‘/etc/my.cnf’? y
修改mysql配置并启动
[[email protected] ~]# vim /etc/my.cnf //（在mysqld模块下添加如下内容）
[[email protected] ~]# sed -i '29 idefault-storage-engine = innodb' /etc/my.cnf //默认的存储引擎
[[email protected] ~]# sed -i '30 iinnodb_file_per_table ' /etc/my.cnf //使用独享的表空间
[[email protected] ~]# sed -i '31 icollation-server = utf8_general_ci' /etc/my.cnf //设置校对标准
[[email protected] ~]# sed -i "32 iinit-connect = 'SET NAMES utf8'" /etc/my.cnf //设置连接的字符集
[[email protected] ~]# sed -i '33 icharacter-set-server = utf8' /etc/my.cnf //设置创建数据库时默认的字符集
开机自启和启动mysql
[[email protected] ~]# systemctl enable mariadb.service
Created symlink from /etc/systemd/system/multi-user.target.wants/mariadb.service to /usr/lib/systemd/system/mariadb.service.
[[email protected] ~]# systemctl start mariadb.service
Job for mariadb.service failed because the control process exited with error code. See "systemctl status mariadb.service" and "journalctl -xe" for details.
启动报错，查看状态：
[[email protected] ~]# systemctl status mariadb.service
mariadb.service - MariaDB 10.1 database server
Loaded: loaded (/usr/lib/systemd/system/mariadb.service; disabled; vendor preset: disabled)
Active: failed (Result: exit-code) since Sun 2016-10-23 15:19:51 CST; 8s ago
Process: 14488 ExecStartPre=/usr/libexec/mysql-prepare-db-dir %n (code=exited, status=1/FAILURE)
Process: 14466 ExecStartPre=/usr/libexec/mysql-check-socket (code=exited, status=0/SUCCESS)
Oct 23 15:19:51 node1 systemd[1]: Starting MariaDB 10.1 database server...
Oct 23 15:19:51 node1 mysql-prepare-db-dir[14488]: Database MariaDB is not ini....
Oct 23 15:19:51 node1 systemd[1]: mariadb.service: control process exited, co...=1
Oct 23 15:19:51 node1 systemd[1]: Failed to start MariaDB 10.1 database server.
Oct 23 15:19:51 node1 systemd[1]: Unit mariadb.service entered failed state.
Oct 23 15:19:51 node1 systemd[1]: mariadb.service failed.
Hint: Some lines were ellipsized, use -l to show in full.
错误原因：未初始化数据库，即未初始化数据库存放数据的目录及运行用户。
解决方法：
[[email protected] ~]# mysql_install_db --datadir="/var/lib/mysql" --user="mysql"
[[email protected] ~]# systemctl start mariadb.service
[[email protected] ~]#systemctl status mariadb.service
设置mysql的密码
[[email protected] ~]# mysql_secure_installation
创建所有组件的库并授权
[[email protected] ~]# mysql -uroot -p123456
执行sql:
CREATE DATABASE keystone;
GRANT ALL PRIVILEGES ON keystone.* TO 'keystone'@'localhost' IDENTIFIED BY 'keystone';
GRANT ALL PRIVILEGES ON keystone.* TO 'keystone'@'%' IDENTIFIED BY 'keystone';
CREATE DATABASE glance;
GRANT ALL PRIVILEGES ON glance.* TO 'glance'@'localhost' IDENTIFIED BY 'glance';
GRANT ALL PRIVILEGES ON glance.* TO 'glance'@'%' IDENTIFIED BY 'glance';
CREATE DATABASE nova;
GRANT ALL PRIVILEGES ON nova.* TO 'nova'@'localhost' IDENTIFIED BY 'nova';
GRANT ALL PRIVILEGES ON nova.* TO 'nova'@'%' IDENTIFIED BY 'nova';
CREATE DATABASE neutron;
GRANT ALL PRIVILEGES ON neutron.* TO 'neutron'@'localhost' IDENTIFIED BY 'neutron';
GRANT ALL PRIVILEGES ON neutron.* TO 'neutron'@'%' IDENTIFIED BY 'neutron';
CREATE DATABASE cinder;
GRANT ALL PRIVILEGES ON cinder.* TO 'cinder'@'localhost' IDENTIFIED BY 'cinder';
GRANT ALL PRIVILEGES ON cinder.* TO 'cinder'@'%' IDENTIFIED BY 'cinder';
3.3 Rabbit消息队列
SOA架构：面向服务的体系结构是一个组件模型，它将应用程序的不同功能单元（称为服务）通过这些服务之间定义良好的接口和契约联系起来。接口是采用中立的方式进行定义的，它应该独立于实现服务的硬件平台、操作系统和编程语言。这使得构建在各种各样的系统中的服务可以使用一种统一和通用的方式进行交互。
在这里Openstack采用了SOA架构方案，结合了SOA架构的松耦合特点，单独组件单独部署，每个组件之间可能互为消费者和提供者，通过消息队列（openstack 支持Rabbitmq，Zeromq，Qpid）进行通信，保证了当某个服务当掉的情况，不至于其他都当掉。
1、启动Rabbitmq
[[email protected] ~]# systemctl enable rabbitmq-server.service
Created symlink from /etc/systemd/system/multi-user.target.wants/rabbitmq-server.service to /usr/lib/systemd/system/rabbitmq-server.service.
[[email protected] ~]# systemctl start rabbitmq-server.service
Job for rabbitmq-server.service failed becausethe control process exited with error code. See "systemctl statusrabbitmq-server.service"and"journalctl -xe" for details
解决方法：
修改主机名和hosts
修改前：
[[email protected] ~]# hostnamectl status
Static hostname: node1.server
Icon name: computer-vm
Chassis: vm
Machine ID: d359f0058624494aa3c144477c6d97b8
Boot ID: bdd32e5df98a41259a441a079c0c44b3
Virtualization: vmware
Operating System: CentOS Linux 7 (Core)
CPE OS Name: cpe:/o:centos:centos:7
Kernel: Linux 3.10.0-327.el7.x86_64
Architecture: x86-64
修改后：
[[email protected] ~]# hostnamectl --static set-hostname node1
[[email protected] ~]# hostnamectl status
Static hostname: node1
Transient hostname: node1.server
Icon name: computer-vm
Chassis: vm
Machine ID: d359f0058624494aa3c144477c6d97b8
Boot ID: bdd32e5df98a41259a441a079c0c44b3
Virtualization: vmware
Operating System: CentOS Linux 7 (Core)
CPE OS Name: cpe:/o:centos:centos:7
Kernel: Linux 3.10.0-327.el7.x86_64
Architecture: x86-64
[[email protected] ~]# cat /etc/hosts
127.0.0.1 localhost localhost.localdomain localhost4 localhost4.localdomain4
::1 localhost localhost.localdomain localhost6 localhost6.localdomain6
192.168.8.150 node1
192.168.8.155 node2
新建Rabbitmq用户并授权
[[email protected] ~]# rabbitmqctl add_user openstack openstack
Creating user "openstack" ...
[[email protected] ~]# rabbitmqctl set_permissions openstack ".*" ".*" ".*"
Setting permissions for user "openstack" in vhost "/" ...
启用Rabbitmq的web管理插件
[[email protected] ~]# rabbitmq-plugins list
[[email protected] ~]# rabbitmq-plugins enable rabbitmq_management
The following plugins have been enabled:
Mochiweb
Webmachine
rabbitmq_web_dispatch
amqp_client
rabbitmq_management_agent
rabbitmq_management
Applying plugin configuration to [email protected] started 6 plugins.
重启Rabbitmq
[[email protected] ~]# systemctl restart rabbitmq-server.service
查看Rabbit的端口，其中5672是服务端口，15672是web管理端口，25672是做集群的端口
[[email protected] ~]# ss -tunlp|grep 5672
tcp LISTEN 0 128 *:25672 *:* users:(("beam",pid=4654,fd=36))
tcp LISTEN 0 128 *:15672 *:* users:(("beam",pid=4654,fd=46))
tcp LISTEN 0 128 :::5672 :::* users:(("beam",pid=4654,fd=45))
在web界面添加openstack用户，设置权限，首次登陆必须使用账号和密码，必须都是guest
role设置为administrator，并设置openstack的密码为openstack
3.4 Keystone组件
名词解释:
User:用户
Tenant:租户，项目
Token:令牌
Role:角色
Service:服务
Endpoint:端点
修改keystone的配置文件
[[email protected] ~]# cp /etc/keystone/keystone.conf /etc/keystone/keystone.conf.ori
[[email protected] ~]# vi /etc/keystone/keystone.conf
12 admin_token = ADMIN //本机配置
1 [DEFAULT]
12 admin_token = 863d35676a5632e846d9 //(生产配置)用作无用户时，创建用户来链接，此内容使用openssl随机产生,此处不用配置
[[email protected] ~]# openssl rand -hex 10
318ff6bd9fdb97ea670e
472 [database]
495 connection = mysql://keystone:[email protected]/keystone //用作链接数据库，三个keysthone分别为keystone组件，keystone用户名，mysql中的keysthone库名
切换到keystone用户，导入keystoe数据库
[[email protected] ~]# su -s /bin/sh -c "keystone-manage db_sync" keystone
[[email protected] ~]# cd /var/log/keystone/
[[email protected] keystone]# ll
total 8
-rw-r--r-- 1 keystone keystone 6964 Sep 17 21:26 keystone.log //（通过切换到keystone用户下导入数据库，当启动的时候会把日志写入到该日志中，如果使用root执行倒库操作，则无法通过keysthone启动keystone程序）
12:admin_token = 318ff6bd9fdb97ea670e
107:verbose = true //开启debug模式
495:connection = mysql://keystone:[email protected]/keystone
1306 [memcache]
1313:servers = localhost:11211 //更改servers标签，填写memcache地址
1709 [revoke]
1718:driver = sql //开启默认sql驱动
1889 [token]
1911:provider = uuid //开启并使用唯一识别码
1916:driver = memcache // (使用用户密码生成token时，存储到memcache中，高性能提供服务)
查看更改结果
[[email protected] ~]# grep -n '^[a-Z]' /etc/keystone/keystone.conf
12:admin_token = 318ff6bd9fdb97ea670e
107:verbose = true
495:connection = mysql://keystone:[email protected]/keystone
1313:servers = 192.168.8.150:11211
1718:driver = sql
1911:provider = uuid
1916:driver = memcache
检查数据库导入结果
[[email protected] keystone]# mysql -uroot -p123456
Welcome to the MariaDB monitor. Commands end with ; or \g.
Your MariaDB connection id is 3
Server version: 5.5.50-MariaDB MariaDB Server
Copyright (c) 2000, 2016, Oracle, MariaDB Corporation Ab and others.
Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.
MariaDB [(none)]> use keystone;
Database changed
MariaDB [keystone]> show tables;
+------------------------+
| Tables_in_keystone |
+------------------------+
| access_token |
| assignment |
| config_register |
| consumer |
| credential |
| domain |
| endpoint |
| endpoint_group |
| federation_protocol |
| group |
| id_mapping |
| identity_provider |
| idp_remote_ids |
| mapping |
| migrate_version |
| policy |
| policy_association |
| project |
| project_endpoint |
| project_endpoint_group |
| region |
| request_token |
| revocation_event |
| role |
| sensitive_config |
| service |
| service_provider |
| token |
| trust |
| trust_role |
| user |
| user_group_membership |
| whitelisted_config |
+------------------------+
33 rows in set (0.00 sec)
添加一个apache的wsgi-keystone配置文件，其中5000端口是提供该服务的，35357是为admin提供管理用的
[[email protected] keystone]# vi /etc/httpd/conf.d/wsgi-keystone.conf
Listen 5000
Listen 35357
<VirtualHost *:5000>
WSGIDaemonProcess keystone-public processes=5 threads=1 user=keystone group=keystone display-name=%{GROUP}
WSGIProcessGroup keystone-public
WSGIScriptAlias / /usr/bin/keystone-wsgi-public
WSGIApplicationGroup %{GLOBAL}
WSGIPassAuthorization On
<IfVersion >= 2.4>
ErrorLogFormat "%{cu}t %M"
</IfVersion>
ErrorLog /var/log/httpd/keystone-error.log
CustomLog /var/log/httpd/keystone-access.log combined
<Directory /usr/bin>
<IfVersion >= 2.4>
Require all granted
</IfVersion>
<IfVersion < 2.4>
Order allow,deny
Allow from all
</IfVersion>
</Directory>
</VirtualHost>
<VirtualHost *:35357>
WSGIDaemonProcess keystone-admin processes=5 threads=1 user=keystone group=keystone display-name=%{GROUP}
WSGIProcessGroup keystone-admin
WSGIScriptAlias / /usr/bin/keystone-wsgi-admin
WSGIApplicationGroup %{GLOBAL}
WSGIPassAuthorization On
<IfVersion >= 2.4>
ErrorLogFormat "%{cu}t %M"
</IfVersion>
ErrorLog /var/log/httpd/keystone-error.log
CustomLog /var/log/httpd/keystone-access.log combined
<Directory /usr/bin>
<IfVersion >= 2.4>
Require all granted
</IfVersion>
<IfVersion < 2.4>
Order allow,deny
Allow from all
</IfVersion>
</Directory>
</VirtualHost>
配置apache的servername,如果不配置servername，会影响keystone服务
[[email protected] keystone]# vi /etc/httpd/conf/httpd.conf
ServerName 192.168.8.150:80
启动memcached，httpd，keystone
[[email protected] keystone]# systemctl enable memcached httpd
Created symlink from /etc/systemd/system/multi-user.target.wants/memcached.service to /usr/lib/systemd/system/memcached.service.
Created symlink from /etc/systemd/system/multi-user.target.wants/httpd.service to /usr/lib/systemd/system/httpd.service.
[[email protected] keystone]# systemctl start memcached httpd
查看httpd占用端口情况
[[email protected] keystone]# ss -tunlp|grep httpd
tcp LISTEN 0 128 :::5000 :::* users:(("httpd",pid=6366,fd=6),("httpd",pid=6365,fd=6),("httpd",pid=6349,fd=6),("httpd",pid=6345,fd=6),("httpd",pid=6344,fd=6),("httpd",pid=6301,fd=6))
tcp LISTEN 0 128 :::80 :::* users:(("httpd",pid=6366,fd=4),("httpd",pid=6365,fd=4),("httpd",pid=6349,fd=4),("httpd",pid=6345,fd=4),("httpd",pid=6344,fd=4),("httpd",pid=6301,fd=4))
tcp LISTEN 0 128 :::35357 :::* users:(("httpd",pid=6366,fd=8),("httpd",pid=6365,fd=8),("httpd",pid=6349,fd=8),("httpd",pid=6345,fd=8),("httpd",pid=6344,fd=8),("httpd",pid=6301,fd=8))
创建用户并连接keystone，在这里可以使用两种方式，通过keystone –help后加参数的方式，或者使用环境变量env的方式，下面就将使用环境变量的方式,分别设置了token，API及控制版本（SOA种很适用）
[[email protected] keystone]# export OS_TOKEN=318ff6bd9fdb97ea670e
[[email protected] keystone]# export OS_URL=http://192.168.8.150:35357/v3
[[email protected] keystone]# export OS_IDENTITY_API_VERSION=3
创建admin项目（project）
[[email protected] keystone]# openstack project create --domain default --description "Admin Project" admin
+-------------+----------------------------------+
| Field | Value |
+-------------+----------------------------------+
| description | Admin Project |
| domain_id | default |
| enabled | True |
| id | 3a5f745d6da0402fb12e699a69b0dd5e |
| is_domain | False |
| name | admin |
| parent_id | None |
+-------------+----------------------------------+
创建admin用户（user）并设置密码（生产环境一定设置一个复杂的）
[[email protected] keystone]# openstack user create --domain default --password-prompt admin
User Password:admin
Repeat User Password:admin
+-----------+----------------------------------+
| Field | Value |
+-----------+----------------------------------+
| domain_id | default |
| enabled | True |
| id | c5f990649b784bea9d7980c71e2e8c31 |
| name | admin |
+-----------+----------------------------------+
创建admin的角色（role）
[[email protected] keystone]# openstack role create admin
+-------+----------------------------------+
| Field | Value |
+-------+----------------------------------+
| id | dc29aaaa19124baa85be2e058d3fb12f |
| name | admin |
+-------+----------------------------------+
把admin用户加到admin项目，赋予admin角色，把角色，项目，用户关联起来
[[email protected] keystone]# openstack role add --project admin --user admin admin
创建一个普通用户demo,demo项目，角色为普通用户（uesr），并把它们关联起来
[[email protected] keystone]# openstack project create --domain default --description "Demo Project" demo
+-------------+----------------------------------+
| Field | Value |
+-------------+----------------------------------+
| description | Demo Project |
| domain_id | default |
| enabled | True |
| id | c2ad2acab9d4438dbc7e17f2fa9aa755 |
| is_domain | False |
| name | demo |
| parent_id | None |
+-------------+----------------------------------+
[[email protected] keystone]# openstack user create --domain default --password=demo demo
+-----------+----------------------------------+
| Field | Value |
+-----------+----------------------------------+
| domain_id | default |
| enabled | True |
| id | a6da61d029de446da9efe7d21af354b8 |
| name | demo |
+-----------+----------------------------------+
[[email protected] keystone]# openstack role create user
+-------+----------------------------------+
| Field | Value |
+-------+----------------------------------+
| id | f52c80593902464c8eff098f14f6a242 |
| name | user |
+-------+----------------------------------+
[[email protected] keystone]# openstack role add --project demo --user demo user
创建一个service的项目，此服务用来管理nova，neutron，glance等组件的服务
[[email protected] keystone]# openstack project create --domain default --description "Service Project" service
+-------------+----------------------------------+
| Field | Value |
+-------------+----------------------------------+
| description | Service Project |
| domain_id | default |
| enabled | True |
| id | 99e62d42a96b4496b28380249a683d6a |
| is_domain | False |
| name | service |
| parent_id | None |
+-------------+----------------------------------+
查看创建的用户,角色，项目
[[email protected] keystone]# openstack user list
+----------------------------------+-------+
| ID | Name |
+----------------------------------+-------+
| a6da61d029de446da9efe7d21af354b8 | demo |
| c5f990649b784bea9d7980c71e2e8c31 | admin |
+----------------------------------+-------+
[[email protected] keystone]# openstack project list
+----------------------------------+---------+
| ID | Name |
+----------------------------------+---------+
| 3a5f745d6da0402fb12e699a69b0dd5e | admin |
| 99e62d42a96b4496b28380249a683d6a | service |
| c2ad2acab9d4438dbc7e17f2fa9aa755 | demo |
+----------------------------------+---------+
[[email protected] keystone]# openstack role list
+----------------------------------+-------+
| ID | Name |
+----------------------------------+-------+
| dc29aaaa19124baa85be2e058d3fb12f | admin |
| f52c80593902464c8eff098f14f6a242 | user |
+----------------------------------+-------+
注册keystone服务，虽然keystone本身是搞注册的，但是自己也需要注册服务
创建keystone认证服务
[[email protected] keystone]# openstack service create --name keystone --description "OpenStack Identity" identity
+-------------+----------------------------------+
| Field | Value |
+-------------+----------------------------------+
| description | OpenStack Identity |
| enabled | True |
| id | ab8dd8ea79a84e86bb7246224df72a73 |
| name | keystone |
| type | identity |
+-------------+----------------------------------+
分别创建三种类型的keystone认证服务endpoint（端点）,分别为public:对外可见，internal内部使用，admin管理使用
[[email protected] ~]# openstack endpoint create --region RegionOne identity public http://192.168.8.150:5000/v2.0
+--------------+----------------------------------+
| Field | Value |
+--------------+----------------------------------+
| enabled | True |
| id | c818a6de667e4edcae65ed22e6d50d8e |
| interface | public |
| region | RegionOne |
| region_id | RegionOne |
| service_id | 6a7bff81961b490ab93090692a227cc6 |
| service_name | keystone |
| service_type | identity |
| url | http://192.168.8.150:5000/v2.0 |
+--------------+----------------------------------+
[[email protected] ~]# openstack endpoint create --region RegionOne identity internal http://192.168.8.150:5000/v2.0
+--------------+----------------------------------+
| Field | Value |
+--------------+----------------------------------+
| enabled | True |
| id | 0e9dfb501cc24a35bfecb27d712f7385 |
| interface | internal |
| region | RegionOne |
| region_id | RegionOne |
| service_id | 6a7bff81961b490ab93090692a227cc6 |
| service_name | keystone |
| service_type | identity |
| url | http://192.168.8.150:5000/v2.0 |
+--------------+----------------------------------+
[[email protected] ~]# openstack endpoint create --region RegionOne identity admin http://192.168.8.150:35357/v2.0
+--------------+----------------------------------+
| Field | Value |
+--------------+----------------------------------+
| enabled | True |
| id | fcbf75c76575414ca8b32e625e26c318 |
| interface | admin |
| region | RegionOne |
| region_id | RegionOne |
| service_id | 6a7bff81961b490ab93090692a227cc6 |
| service_name | keystone |
| service_type | identity |
| url | http://192.168.8.150:35357/v2.0 |
+--------------+----------------------------------+
查看创建的keystone认证服务endpoint（端点）
[[email protected] ~]# openstack endpoint delete ID //删除服务端点
[[email protected] ~]# openstack endpoint list
+----------------------------------+-----------+--------------+--------------+---------+-----------+---------------------------------+
| ID | Region | Service Name | Service Type | Enabled | Interface | URL |
+----------------------------------+-----------+--------------+--------------+---------+-----------+---------------------------------+
| 0e9dfb501cc24a35bfecb27d712f7385 | RegionOne | keystone | identity | True | internal | http://192.168.8.150:5000/v2.0 |
| c818a6de667e4edcae65ed22e6d50d8e | RegionOne | keystone | identity | True | public | http://192.168.8.150:5000/v2.0 |
| fcbf75c76575414ca8b32e625e26c318 | RegionOne | keystone | identity | True | admin | http://192.168.8.150:35357/v2.0 |
+----------------------------------+-----------+--------------+--------------+---------+-----------+---------------------------------+
链接到keystone，请求token，在这里由于已经添加了用户名和密码，就不在使用token，所以就一定要取消环境变量了
[[email protected] ~]# unset OS_TOKEN
[[email protected] ~]# unset OS_URL
[[email protected] ~]# openstack --os-auth-url http://192.168.8.150:35357/v3 --os-project-domain-id default --os-user-domain-id default --os-project-name admin --os-username admin --os-auth-type password token issue
Password: admin
+------------+----------------------------------+
| Field | Value |
+------------+----------------------------------+
| expires | 2016-10-23T10:28:10.690753Z |
| id | 6c1f2679cec0498dbd47067b022215fa |
| project_id | 3ca500e1969c4207890c139d8da26499 |
| user_id | be1a298005f040b0aae93c246090ffc4 |
+------------+----------------------------------+
配置admin和demo用户的环境变量，并添加执行权限，以后执行命令,直接source一下就行了
[[email protected] ~]# cat admin-openrc.sh
export OS_PROJECT_DOMAIN_ID=default
export OS_USER_DOMAIN_ID=default
export OS_PROJECT_NAME=admin
export OS_TENANT_NAME=admin
export OS_USERNAME=admin
export OS_PASSWORD=admin
export OS_AUTH_URL=http://192.168.8.150:35357/v3
export OS_IDENTITY_API_VERSION=3
[[email protected] ~]# cat demo-openrc.sh
export OS_PROJECT_DOMAIN_ID=default
export OS_USER_DOMAIN_ID=default
export OS_PROJECT_NAME=demo
export OS_TENANT_NAME=demo
export OS_USERNAME=demo
export OS_PASSWORD=demo
export OS_AUTH_URL=http://192.168.8.150:5000/v3
export OS_IDENTITY_API_VERSION=3
[[email protected] ~]# chmod +x admin-openrc.sh
[[email protected] ~]# chmod +x demo-openrc.sh
[[email protected] ~]# source admin-openrc.sh
[[email protected] ~]# openstack token issue
+------------+----------------------------------+
| Field | Value |
+------------+----------------------------------+
| expires | 2016-10-23T10:38:22.810502Z |
| id | 8621d9c2386a448d88182517692718a2 |
| project_id | 3ca500e1969c4207890c139d8da26499 |
| user_id | be1a298005f040b0aae93c246090ffc4 |
+------------+----------------------------------+
3.5 Glance部署
修改glance-api和glance-registry的配置文件，同步数据库
[[email protected] ~]# ll /etc/glance/
total 140
-rw-r-----. 1 root glance 54203 Mar 7 2016 glance-api.conf
-rw-r-----. 1 root glance 10274 Mar 7 2016 glance-cache.conf
-rw-r-----. 1 root glance 43522 Mar 7 2016 glance-registry.conf
-rw-r-----. 1 root glance 15220 Mar 7 2016 glance-scrubber.conf
drwxr-xr-x. 2 root root 4096 Sep 17 21:46 metadefs
-rw-r-----. 1 root glance 1311 Dec 21 2015 policy.json
-rw-r-----. 1 root glance 1279 Dec 21 2015 schema-image.json
[[email protected] glance]# vi glance-api.conf
515 [database]
538 connection=mysql://glance:[email protected]/glance
[[email protected] glance]# vi glance-registry.conf
340 [database]
363 connection=mysql://glance:[email protected]/glance
[[email protected] glance]# su -s /bin/sh -c "glance-manage db_sync" glance
No handlers could be found for logger "oslo_config.cfg"
/usr/lib64/python2.7/site-packages/sqlalchemy/engine/default.py:450: Warning: Duplicate index 'ix_image_properties_image_id_name' defined on the table 'glance.image_properties'. This is deprecated and will be disallowed in a future release.
cursor.execute(statement, parameters) //（可以忽略）
检查导入glance库的表情况
MariaDB [(none)]> use glance;
Database changed
MariaDB [glance]> show tables;
+----------------------------------+
| Tables_in_glance |
+----------------------------------+
| artifact_blob_locations |
| artifact_blobs |
| artifact_dependencies |
| artifact_properties |
| artifact_tags |
| artifacts |
| image_locations |
| image_members |
| image_properties |
| image_tags |
| images |
| metadef_namespace_resource_types |
| metadef_namespaces |
| metadef_objects |
| metadef_properties |
| metadef_resource_types |
| metadef_tags |
| migrate_version |
| task_info |
| tasks |
+----------------------------------+
20 rows in set (0.00 sec)
配置glance连接keystone，对于keystone，每个服务都要有一个用户连接keystone
[[email protected] ~]# source admin-openrc.sh
[[email protected] ~]# openstack user create --domain default --password=glance glance
+-----------+----------------------------------+
| Field | Value |
+-----------+----------------------------------+
| domain_id | default |
| enabled | True |
| id | 9e31255125fa4885a5786c78c3b8b1c5 |
| name | glance |
+-----------+----------------------------------+
[[email protected] ~]# openstack role add --project service --user glance admin
修改glance-api配置文件，结合keystone和mysql
[[email protected] glance]# vim glance-api.conf
973 [keystone_authtoken]
978 auth_uri = http://192.168.8.150:5000
979 auth_url = http://192.168.8.150:35357
980 auth_plugin = password
981 project_domain_id = default
982 user_domain_id = default
983 project_name = service
984 username = glance
985 password = glance
1475 [paste_deploy]
1485 flavor=keystone
1 [DEFAULT]
363 verbose=True //打开debug
491 notification_driver = noop //镜像服务不需要使用消息队列
630 [glance_store]
642 default_store=file //镜像存放成文件
701 filesystem_store_datadir=/var/lib/glance/images/ //镜像存放位置
修改glance-registry配置文件，结合keystone和mysql
[[email protected] glance]# vim glance-registry.conf
1 [DEFAULT]
188:verbose=True
316:notification_driver =noop
762 [keystone_authtoken]
767 auth_uri = http://192.168.8.150:5000
768 auth_url = http://192.168.8.150:35357
769 auth_plugin = password
770 project_domain_id = default
771 user_domain_id = default
772 project_name = service
773 username = glance
774 password = glance
1246 [paste_deploy]
1256:flavor=keystone
检查glance修改过的配置
[[email protected] ~]# grep -n '^[a-z]' /etc/glance/glance-api.conf
363:verbose=True
491:notification_driver = noop
538:connection=mysql://glance:[email protected]/glance
642:default_store=file
701:filesystem_store_datadir=/var/lib/glance/images/
978:auth_uri = http://192.168.8.150:5000
979:auth_url = http://192.168.8.150:35357
980:auth_plugin = password
981:project_domain_id = default
982:user_domain_id = default
983:project_name = service
984:username = glance
985:password = glance
1485:flavor=keystone
[[email protected] ~]# grep -n '^[a-z]' /etc/glance/glance-registry.conf
188:verbose=True
363:connection=mysql://glance:[email protected]/glance
767:auth_uri = http://192.168.8.150:5000
768:auth_url = http://192.168.8.150:35357
769:auth_plugin = password
770:project_domain_id = default
771:user_domain_id = default
772:project_name = service
773:username = glance
774:password = glance
1256:flavor=keystone
对glance设置开机启动并启动glance服务
[[email protected] ~]# systemctl enable openstack-glance-api
Created symlink from /etc/systemd/system/multi-user.target.wants/openstack-glance-api.service to /usr/lib/systemd/system/openstack-glance-api.service.
[[email protected] ~]# systemctl enable openstack-glance-registry
Created symlink from /etc/systemd/system/multi-user.target.wants/openstack-glance-registry.service to /usr/lib/systemd/system/openstack-glance-registry.service.
[[email protected] ~]# systemctl start openstack-glance-api
[[email protected] ~]# systemctl start openstack-glance-registry
查看galnce占用端口情况，其中9191是registry占用端口，9292是api占用端口
[[email protected] ~]# ss -tunlp|egrep "9191|9292"
tcp LISTEN 0 128 *:9292 *:* users:(("glance-api",pid=9654,fd=4),("glance-api",pid=9643,fd=4))
tcp LISTEN 0 128 *:9191 *:* users:(("glance-registry",pid=9673,fd=4),("glance-registry",pid=9662,fd=4))
使glance服务在keystone上注册，才可以允许其他服务调用glance
[[email protected] ~]# source admin-openrc.sh
[[email protected] ~]# openstack service create --name glance --description "OpenStack Image service" image
+-------------+----------------------------------+
| Field | Value |
+-------------+----------------------------------+
| description | OpenStack Image service |
| enabled | True |
| id | 948202ce1f0e4d9b8d987173a2b6eb30 |
| name | glance |
| type | image |
+-------------+----------------------------------+
[[email protected] ~]# openstack endpoint create --region RegionOne image public http://192.168.8.150:9292
+--------------+----------------------------------+
| Field | Value |
+--------------+----------------------------------+
| enabled | True |
| id | 46a2585071bc4e7c90f7535ad39d6202 |
| interface | public |
| region | RegionOne |
| region_id | RegionOne |
| service_id | 948202ce1f0e4d9b8d987173a2b6eb30 |
| service_name | glance |
| service_type | image |
| url | http://192.168.8.150:9292 |
+--------------+----------------------------------+
[[email protected] ~]# openstack endpoint create --region RegionOne image internal http://192.168.8.150:9292
+--------------+----------------------------------+
| Field | Value |
+--------------+----------------------------------+
| enabled | True |
| id | 97e6f73ed6634a0bb99c759115ab1635 |
| interface | internal |
| region | RegionOne |
| region_id | RegionOne |
| service_id | 948202ce1f0e4d9b8d987173a2b6eb30 |
| service_name | glance |
| service_type | image |
| url | http://192.168.8.150:9292 |
+--------------+----------------------------------+
[[email protected] ~]# openstack endpoint create --region RegionOne image admin http://192.168.8.150:9292
+--------------+----------------------------------+
| Field | Value |
+--------------+----------------------------------+
| enabled | True |
| id | 59512d306e9845c1962e6212d582dc1c |
| interface | admin |
| region | RegionOne |
| region_id | RegionOne |
| service_id | 948202ce1f0e4d9b8d987173a2b6eb30 |
| service_name | glance |
| service_type | image |
| url | http://192.168.8.150:9292 |
+--------------+----------------------------------+
在admin和demo中加入glance的环境变量,告诉其他服务glance使用的环境变量，一定要在admin-openrc.sh的路径下执行
[[email protected] ~]# echo "export OS_IMAGE_API_VERSION=2" | tee -a admin-openrc.sh demo-openrc.sh
export OS_IMAGE_API_VERSION=2
[[email protected] ~]# tail -1 admin-openrc.sh
export OS_IMAGE_API_VERSION=2
[[email protected] ~]# tail -1 demo-openrc.sh
export OS_IMAGE_API_VERSION=2
如果出现以下情况，表示glance配置成功，由于没有镜像，所以看不到
[[email protected] ~]# glance image-list
+----+------+
| ID | Name |
+----+------+
+----+------+
下载一个镜像
[[email protected] ~]# wget http://download.cirros-cloud.net/0.3.4/cirros-0.3.4-x86_64-disk.img
--2015-12-17 02:12:55-- http://download.cirros-cloud.net/0.3.4/cirros-0.3.4-x86_64-disk.img
Resolving download.cirros-cloud.net (download.cirros-cloud.net)... 69.163.241.114
Connecting to download.cirros-cloud.net (download.cirros-cloud.net)|69.163.241.114|:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: 13287936 (13M) [text/plain]
Saving to: ‘cirros-0.3.4-x86_64-disk.img’
100%[======================================>] 13,287,936 127KB/s in 71s
2015-12-17 02:14:08 (183 KB/s) - ‘cirros-0.3.4-x86_64-disk.img’ saved [13287936/13287936]
上传镜像到glance,要在上一步所下载的镜像当前目录执行
[[email protected] ~]# glance image-create --name "cirros" --file cirros-0.3.4-x86_64-disk.img --disk-format qcow2 --container-format bare --visibility public --progress
403 Forbidden: You are not authorized to complete this action. (HTTP 403)
意思是此次上传镜像未被授权。
解决方法：
经过查看分析日志以及错误提示，得出配置文件错误，正确的配置文件如下：
[[email protected] ~]# grep -n '^[a-z]' /etc/glance/glance-api.conf
2:notification_driver=noop
3:verbose=True
540:connection=mysql://glance:[email protected]/glance
982:auth_uri=http://192.168.8.150:5000
983:auth_url=http://192.168.8.150:35357
984:auth_plugin=password
985:project_domain_id=default
986:user_domain_id=default
987:project_name=service
988:username=glance
989:password=glance
990:default_store=file
991:filesystem_store_datadir=/var/lib/glance/images/
1489:flavor=keystone
[[email protected] ~]# systemctl restart openstack-glance-api
[[email protected] ~]# systemctl restart openstack-glance-registry
[[email protected] ~]# su -s /bin/sh -c "glance-manage db_sync" glance
No handlers could be found for logger "oslo_config.cfg"
[[email protected] ~]# source admin-openrc.sh [[email protected] ~]# glance image-create --name "cirros" --file=cirros-0.3.4-x86_64-disk.img --disk-format=qcow2 --container-format=bare --visibility public --progress
[=============================>] 100%
+------------------+--------------------------------------+
| Property | Value |
+------------------+--------------------------------------+
| checksum | ee1eca47dc88f4879d8a229cc70a07c6 |
| container_format | bare |
| created_at | 2016-09-19T15:19:28Z |
| disk_format | qcow2 |
| id | c25c409d-6e7d-4855-9322-41cc1141c350 |
| min_disk | 0 |
| min_ram | 0 |
| name | cirros |
| owner | 4c1ee18980c64098a1e86fb43a2429bb |
| protected | False |
| size | 13287936 |
| status | active |
| tags | [] |
| updated_at | 2016-09-19T15:19:35Z |
| virtual_size | None |
| visibility | public |
+------------------+--------------------------------------+
由此得出，同样的配置文件，配置的位置不一样，就会出现以上故障。就是说必须要在相应的“模块”下配置才行。
例如，如下：
a.修改数据库连接（直接添加即可）
1. [database]
2. ...
3. connection =mysql://glance:[email protected]/glance
b.在 [keystone_authtoken] 和 [paste_deploy] 部分, 修改配置: 添加如下内容，切记不要keystone_authtoken标记重复
1. [keystone_authtoken]
2. ...
3. auth_uri =http://controller:5000/v2.0
4. identity_uri =http://controller:35357
5. admin_tenant_name = service
6. admin_user = glance
7. admin_password = GLANCE_PASS
c.修改flavor
1. [paste_deploy]
2. ...
3. flavor = keystone
d.修改 [glance_store] 部分，配置本地文件存储及存储路径
1. [glance_store]
2. ...
3. default_store = file
4. filesystem_store_datadir =/var/lib/glance/images/
e.在[DEFAULT]部分，配置 noop通知驱动
1.
2. [DEFAULT]
3. ...
4. notification_driver = noop
遥测提供Image 服务配置
f.可选，帮助排除定位错误，使日志记录在[DEFAULT]部分
1. [DEFAULT]
2. ...
3. verbose = True
查看上传镜像
[[email protected] ~]# glance image-list
+--------------------------------------+--------+
| ID | Name |
+--------------------------------------+--------+
| c25c409d-6e7d-4855-9322-41cc1141c350 | cirros |
+--------------------------------------+--------+
[[email protected] ~]# cd /var/lib/glance/images/
[[email protected] images]# ls
c25c409d-6e7d-4855-9322-41cc1141c350 (和上述ID一致)
[[email protected] ~]# file /var/lib/glance/images/cca2b796-6c5e-4746-b697-f126604f7063
/var/lib/glance/images/cca2b796-6c5e-4746-b697-f126604f7063: QEMU QCOW Image (v2), 41126400 bytes
转载于:https://blog.51cto.com/linuxzkq/1876982