Mongodb -(3) replica set+sharding
---何旭东
目录
分片集搭建...................................................................................................................... 1
生态系统.......................................................................................................................... 2
环境介绍.......................................................................................................................... 2
需求.......................................................................................................................... 2
实际拓扑................................................................................................................... 2
机器端口................................................................................................................... 2
机器功能................................................................................................................... 3
机器IP...................................................................................................................... 3
基础环境................................................................................................................... 4
配置文件.......................................................................................................................... 4
机器一...................................................................................................................... 4
机器二...................................................................................................................... 6
机器三...................................................................................................................... 8
启动................................................................................................................................. 9
第一台..................................................................................................................... 10
第二台..................................................................................................................... 10
第三台..................................................................................................................... 10
启动路由........................................................................................................................ 10
第一台初始化分片及结果......................................................................................... 10
第二台初始化分片及结果......................................................................................... 12
第三台机器添加分片................................................................................................ 13
切分数据.......................................................................................................... 13
结果查看........................................................................................................................ 13
第一个片................................................................................................................. 14
第二个片................................................................................................................. 16
优化............................................................................................................................... 18
常用命令行..................................................................................................................... 18
http://shanqiangwu.blog.51cto.com/8067564/1653968
感谢
感谢大家的帮助及引到，碰到问题不要用百度。找资料去些运维网站
官网网站
2015年7月17日
生态系统
数据服务节点（mongod）
路由节点（mongos）
配置节点（mongod）--config server
投票节点/表决节点（mongod） --arbiter server
环境介绍
需求
三台机器
数据节点，sharding 2片
每个sharding为Replic Set方式，冗余度3（主、从、延迟从）
mongos（路由节点），数量3
config server(配置服务器节点)，数量3
arbiter server(表决节点)，数量6
实际拓扑
机器端口
机器 |
数据节点 |
配置节点 |
表决节点 |
路由节点 |
||
第一台 |
shard11:28017 |
shard21:28018 |
config1:20000 |
arbiter1:28031 |
arbiter2:28032 |
mongos1: 28885 |
第二台 |
shard12:28017 |
Shard22:28018 |
config2:20000 |
arbiter1:28031 |
arbiter2:28032 |
mongos2:28885 |
第三台 |
shard13:28017 |
Shard23:28018 |
config3:20000 |
arbiter1:28031 |
arbiter2:28032 |
mongos3: 28885 |
机器功能
机器 |
端口 |
功能 |
192.168.200.185 |
28017 |
'shard1' PRIMARY |
主机：mongohost1 |
28018 |
‘shard2’ SECONDARY |
20000 |
配置节点 |
|
28031 |
'shard1' ARBITER |
|
28032 |
‘shard2’ ARBITER |
|
28885 |
路由节点 |
|
41000 |
启动路由 生成此端口 |
|
192.168.200.186 |
28017 |
'shard1' SECONDARY |
主机：mongohost2 |
28018 |
‘shard2’ PRIMARY |
20000 |
配置节点 |
|
28031 |
'shard1' ARBITER |
|
28032 |
‘shard2’ ARBITER |
|
28885 |
路由节点 |
|
41000 |
启动路由 生成此端口 |
|
192.168.200.187 |
28017 |
'shard1' SECONDARY |
主机：mongohost3 |
28018 |
‘shard2’ SECONDARY |
20000 |
配置节点 |
|
28031 |
'shard1' ARBITER |
|
28032 |
‘shard2’ ARBITER |
|
28885 |
路由节点 |
|
41000 |
启动路由 生成此端口 |
机器IP
机器 |
IP地址 |
主机名 |
第一台 |
192.168.200.185 |
mongohost1 |
第二台 |
192.168.200.185 |
mongohost2 |
第三台 |
192.168.200.187 |
mongohost3 |
基础环境
iptables 关闭
selinux 关闭
hosts 地址表添加
配置文件
机器一
mkdir /usr/local/mongodb/shard11
mkdir /usr/local/mongodb/shard21
mkdir /usr/local/mongodb/arbiter1
mkdir /usr/local/mongodb/arbiter2/
mkdir /usr/local/mongodb/config/
mkdir /usr/local/mongodb/logs/
cat > /etc/shard11.conf <<EOF
shardsvr=true
port=28017
dbpath = /usr/local/mongodb/shard11/
logpath = /usr/local/mongodb/logs/shard11.log
logappend=true
fork=true
nojournal=true
rest=true
replSet=shard1
oplogSize=2048
EOF
cat > /etc/shard21.conf <<EOF
shardsvr=true
port=28018
dbpath = /usr/local/mongodb/shard21/
logpath =/usr/local/mongodb/logs/shard21.log
logappend=true
fork=true
nojournal=true
rest=true
replSet=shard2
oplogSize=2048
EOF
cat > /etc/config1.conf <<EOF
configsvr=true
port=20000
dbpath = /usr/local/mongodb/config/
logpath =/usr/local/mongodb/logs/config1.log
logappend=true
fork=true
nojournal=true
EOF
cat > /etc/arbiter1.conf <<EOF
shardsvr=true
replSet=shard1
port=28031
dbpath = /usr/local/mongodb/arbiter1/
logpath =/usr/local/mongodb/logs/arbiter1.log
oplogSize=100
logappend=true
fork=true
nojournal=true
rest=true
EOF
cat > /etc/arbiter2.conf <<EOF
shardsvr=true
replSet=shard2
port=28032
dbpath = /usr/local/mongodb/arbiter2/
logpath =/usr/local/mongodb/logs/arbiter2.log
oplogSize=100
logappend=true
fork=true
nojournal=true
rest=true
EOF
cat > /etc/mongos1.conf <<EOF
configdb=mongohost1:20000,mongohost2:20000,mongohost3:20000
chunkSize=100
port=28885
logpath =/usr/local/mongodb/logs/mongos1.log
logappend=true
fork=true
EOF
机器二
mkdir /usr/local/mongodb/shard12
mkdir /usr/local/mongodb/shard22
mkdir /usr/local/mongodb/arbiter1/
mkdir /usr/local/mongodb/arbiter2/
mkdir /usr/local/mongodb/cofig/
mkdir /usr/local/mongodb/logs/
cat > /etc/shard12.conf <<EOF
shardsvr=true
port=28017
dbpath = /usr/local/mongodb/shard12/
logpath = /usr/local/mongodb/logs/shard12.log
logappend=true
fork=true
nojournal=true
rest=true
replSet=shard1
oplogSize=2048
EOF
cat > /etc/shard22.conf <<EOF
shardsvr=true
port=28018
dbpath = /usr/local/mongodb/shard22/
logpath = /usr/local/mongodb/logs/shard22.log
logappend=true
fork=true
nojournal=true
rest=true
replSet=shard2
oplogSize=2048
EOF
cat > /etc/config2.conf <<EOF
configsvr=true
port=20000
dbpath = /usr/local/mongodb/config/
logpath = /usr/local/mongodb/logs/config2.log
logappend=true
fork=true
nojournal=true
EOF
cat > /etc/arbiter1.conf <<EOF
shardsvr=true
replSet=shard1
port=28031
dbpath = /usr/local/mongodb/arbiter1/
logpath =/usr/local/mongodb/logs/arbiter1.log
oplogSize=100
logappend=true
fork=true
nojournal=true
rest=true
EOF
cat > /etc/arbiter2.conf <<EOF
shardsvr=true
replSet=shard2
port=28032
dbpath = /usr/local/mongodb/arbiter2/
logpath =/usr/local/mongodb/logs/arbiter2.log
oplogSize=100
logappend=true
fork=true
nojournal=true
rest=true
EOF
cat > /etc/mongos2.conf <<EOF
configdb=mongohost1:20000,mongohost2:20000,mongohost3:20000
chunkSize=100
port=28885
logpath = /usr/local/mongodb/logs/mongos2.log
logappend=true
fork=true
EOF
机器三
mkdir /usr/local/mongodb/shard13
mkdir /usr/local/mongodb/shard23
mkdir /usr/local/mongodb/arbiter1/
mkdir /usr/local/mongodb/arbiter2/
mkdir /usr/local/mongodb/cofig/
mkdir /usr/local/mongodb/logs/
cat > /etc/shard13.conf <<EOF
shardsvr=true
port=28017
dbpath = /usr/local/mongodb/shard13/
logpath = /usr/local/mongodb/logs/shard13.log
logappend=true
fork=true
nojournal=true
rest=true
replSet=shard1
oplogSize=2048
EOF
cat > /etc/shard23.conf <<EOF
shardsvr=true
port=28018
dbpath = /usr/local/mongodb/shard23/
logpath = /usr/local/mongodb/logs/shard23.log
logappend=true
fork=true
nojournal=true
rest=true
replSet=shard2
oplogSize=2048
EOF
cat > /etc/config3.conf <<EOF
configsvr=true
port=20000
dbpath = /usr/local/mongodb/config/
logpath = /usr/local/mongodb/logs/config3.log
logappend=true
fork=true
nojournal=true
EOF
cat > /etc/arbiter1.conf <<EOF
shardsvr=true
replSet=shard1
port=28031
dbpath = /usr/local/mongodb/arbiter1/
logpath =/usr/local/mongodb/logs/arbiter1.log
oplogSize=100
logappend=true
fork=true
nojournal=true
rest=true
EOF
cat > /etc/arbiter2.conf <<EOF
shardsvr=true
replSet=shard2
port=28032
dbpath = /usr/local/mongodb/arbiter2/
logpath =/usr/local/mongodb/logs/arbiter2.log
oplogSize=100
logappend=true
fork=true
nojournal=true
rest=true
EOF
cat > /etc/mongos3.conf <<EOF
configdb=mongohost1:20000,mongohost2:20000,mongohost3:20000
chunkSize=100
port=28885
logpath = /usr/local/mongodb/logs/mongos3.log
logappend=true
fork=true
EOF
启动
第一台
mongod -f /etc/shard11.conf
mongod -f /etc/shard21.conf
mongod -f /etc/arbiter1.conf
mongod -f /etc/arbiter2.conf
mongod -f /etc/config1.conf
mongos -f /etc/mongos1.conf
第二台
mongod -f /etc/shard12.conf
mongod -f /etc/shard22.conf
mongod -f /etc/arbiter1.conf
mongod -f /etc/arbiter2.conf
mongod -f /etc/config2.conf
mongos -f /etc/mongos2.conf
第三台
mongod -f /etc/shard13.conf
mongod -f /etc/shard23.conf
mongod -f /etc/arbiter1.conf
mongod -f /etc/arbiter2.conf
mongod -f /etc/config3.conf
mongos -f /etc/mongos3.conf
启动路由
mongos --configdb mongohost3:20000,mongohost2:20000,mongohost1:20000--port 41000 --chunkSize 100 --logpath /usr/local/mongodb/logs/mongos.log--logappend --fork
第一台初始化分片及结果
mongo mongohost1:28017/admin
> config = {_id:'shard1', members:[{_id: 0, host: 'mongohost1:28017'},{_id: 1, host: 'mongohost2:28017'},{"_id" : 2,"host":"mongohost3:28017",slaveDelay :7200 ,priority:0},{_id: 3, host: 'mongohost1:28031', arbiterOnly : true},{_id:4, host: 'mongohost2:28031', arbiterOnly : true},{_id: 5, host:'mongohost3:28031', arbiterOnly : true}]};
{
"_id" : "shard1",
"members" : [
{
"_id" : 0,
"host" :"mongohost1:28017"
},
{
"_id" : 1,
"host" :"mongohost2:28017"
},
{
"_id" : 2,
"host" :"mongohost3:28017",
"slaveDelay": 7200,
"priority" :0
},
{
"_id" : 3,
"host" :"mongohost1:28031",
"arbiterOnly": true
},
{
"_id" : 4,
"host" :"mongohost2:28031",
"arbiterOnly": true
},
{
"_id" : 5,
"host" :"mongohost3:28031",
"arbiterOnly": true
}
]
}
> rs.initiate(config);
{
"info" : "Config now saved locally. Should come online in about a minute.",
"ok" : 1
}
>
第二台初始化分片及结果
mongo mongohost2:28018/admin
> config = {_id:'shard2', members:[{_id: 0, host: 'mongohost2:28018'},{_id: 1, host: 'mongohost1:28018'},{"_id" : 2,"host":"mongohost3:28018",slaveDelay :7200 ,priority:0},{_id: 3, host: 'mongohost1:28032', arbiterOnly : true},{_id:4, host: 'mongohost2:28032', arbiterOnly : true},{_id: 5, host:'mongohost3:28032', arbiterOnly : true}]};
{
"_id" : "shard2",
"members" : [
{
"_id" : 0,
"host" :"mongohost2:28018"
},
{
"_id" : 1,
"host" :"mongohost1:28018"
},
{
"_id" : 2,
"host" :"mongohost3:28018",
"slaveDelay": 7200,
"priority" :0
},
{
"_id" : 3,
"host" :"mongohost1:28032",
"arbiterOnly": true
},
{
"_id" : 4,
"host" :"mongohost2:28032",
"arbiterOnly": true
},
{
"_id" : 5,
"host" :"mongohost3:28032",
"arbiterOnly": true
}
]
}
> rs.initiate(config);
{
"info" : "Config now saved locally. Should come online in about a minute.",
"ok" : 1
}
>
第三台机器添加分片
[[email protected] logs]# mongomongohost3:28885/admin
MongoDB shell version: 2.6.10
connecting to: mongohost3:28885/admin
mongos>db.runCommand({"addshard" :"shard1/mongohost1:28017,mongohost2:28017"})
{ "shardAdded" :"shard1", "ok" : 1 }
mongos> db.runCommand({"addshard": "shard2/mongohost2:28018,mongohost1:28018"})
{ "shardAdded" :"shard2", "ok" : 1 }
mongos>
切分数据
db.runCommand({ "enablesharding" : "im_offline_msg"})
db.runCommand({ shardcollection : "step.step",key : {_id: 1} } )
db.runCommand( { "enablesharding": "xx"}) 开启库的分片功能
db.runCommand( { shardcollection :"xx",key : {_id: 1} } )开启表的分片功能，key表示shard key
结果查看
mongos> db.printShardingStatus();
--- Sharding Status ---
sharding version: {
"_id" : 1,
"version" : 4,
"minCompatibleVersion" : 4,
"currentVersion" : 5,
"clusterId" : ObjectId("55ac8d1e55715e4e18194a14")
}
shards:
{ "_id" :"shard1", "host" :"shard1/mongohost1:28017,mongohost2:28017" }
{ "_id" :"shard2", "host" :"shard2/mongohost1:28018,mongohost2:28018" }
databases:
{ "_id" :"admin", "partitioned" : false, "primary" : "config" }
{ "_id" :"im_offline_msg", "partitioned" : true, "primary" : "shard1" }
{ "_id" :"step", "partitioned" : false, "primary" : "shard1" }
mongos>
第一个片
shard1:PRIMARY> rs.status()
{
"set" : "shard1",
"date" : ISODate("2015-07-17T11:11:12Z"),
"myState" : 1,
"members" : [
{
"_id" : 0,
"name" :"mongohost1:28017",
"health" : 1,
"state" : 1,
"stateStr" :"PRIMARY",
"uptime" :4498,
"optime" :Timestamp(1437129067, 2418),
"optimeDate": ISODate("2015-07-17T10:31:07Z"),
"electionTime" : Timestamp(1437127220, 1),
"electionDate" : ISODate("2015-07-17T10:00:20Z"),
"self" : true
},
{
"_id" : 1,
"name" :"mongohost2:28017",
"health" : 1,
"state" : 2,
"stateStr" :"SECONDARY",
"uptime" :4262,
"optime" :Timestamp(1437129067, 2418),
"optimeDate": ISODate("2015-07-17T10:31:07Z"),
"lastHeartbeat" : ISODate("2015-07-17T11:11:11Z"),
"lastHeartbeatRecv" :ISODate("2015-07-17T11:11:10Z"),
"pingMs" : 0,
"syncingTo" :"mongohost1:28017"
},
{
"_id" : 2,
"name" :"mongohost3:28017",
"health" : 1,
"state" : 2,
"stateStr" :"SECONDARY",
"uptime" : 4262,
"optime" :Timestamp(1437127210, 1),
"optimeDate": ISODate("2015-07-17T10:00:10Z"),
"lastHeartbeat" : ISODate("2015-07-17T11:11:11Z"),
"lastHeartbeatRecv" :ISODate("2015-07-17T11:11:11Z"),
"pingMs" : 0,
"syncingTo" :"mongohost1:28017"
},
{
"_id" : 3,
"name" :"mongohost1:28031",
"health" : 1,
"state" : 7,
"stateStr" :"ARBITER",
"uptime" :4262,
"lastHeartbeat" : ISODate("2015-07-17T11:11:12Z"),
"lastHeartbeatRecv" :ISODate("2015-07-17T11:11:12Z"),
"pingMs" : 0
},
{
"_id" : 4,
"name" :"mongohost2:28031",
"health" : 1,
"state" : 7,
"stateStr" :"ARBITER",
"uptime" :4262,
"lastHeartbeat" : ISODate("2015-07-17T11:11:11Z"),
"lastHeartbeatRecv" :ISODate("2015-07-17T11:11:11Z"),
"pingMs" : 0
},
{
"_id" : 5,
"name" :"mongohost3:28031",
"health" :1,
"state" : 7,
"stateStr" :"ARBITER",
"uptime" :4260,
"lastHeartbeat" : ISODate("2015-07-17T11:11:11Z"),
"lastHeartbeatRecv" :ISODate("2015-07-17T11:11:11Z"),
"pingMs" : 0
}
],
"ok" : 1
}
shard1:PRIMARY>
第二个片
shard2:SECONDARY> rs.status()
{
"set" : "shard2",
"date" : ISODate("2015-07-17T11:09:51Z"),
"myState" : 2,
"syncingTo" : "mongohost2:28018",
"members" : [
{
"_id" : 0,
"name" :"mongohost2:28018",
"health" : 1,
"state" : 1,
"stateStr" :"PRIMARY",
"uptime" :4074,
"optime" :Timestamp(1437127314, 1),
"optimeDate": ISODate("2015-07-17T10:01:54Z"),
"lastHeartbeat" : ISODate("2015-07-17T11:09:51Z"),
"lastHeartbeatRecv" :ISODate("2015-07-17T11:09:50Z"),
"pingMs" : 0,
"electionTime" : Timestamp(1437127325, 1),
"electionDate" : ISODate("2015-07-17T10:02:05Z")
},
{
"_id" : 1,
"name" :"mongohost1:28018",
"health" : 1,
"state" : 2,
"stateStr" :"SECONDARY",
"uptime" :14970,
"optime" :Timestamp(1437127314, 1),
"optimeDate" :ISODate("2015-07-17T10:01:54Z"),
"self" : true
},
{
"_id" : 2,
"name" :"mongohost3:28018",
"health" : 1,
"state" : 2,
"stateStr" :"SECONDARY",
"uptime" :4074,
"optime" :Timestamp(1437127314, 1),
"optimeDate": ISODate("2015-07-17T10:01:54Z"),
"lastHeartbeat" : ISODate("2015-07-17T11:09:51Z"),
"lastHeartbeatRecv" :ISODate("2015-07-17T11:09:50Z"),
"pingMs" : 0,
"syncingTo" :"mongohost2:28018"
},
{
"_id" : 3,
"name" :"mongohost1:28032",
"health" : 1,
"state" : 7,
"stateStr" :"ARBITER",
"uptime" :4076,
"lastHeartbeat" : ISODate("2015-07-17T11:09:50Z"),
"lastHeartbeatRecv" :ISODate("2015-07-17T11:09:51Z"),
"pingMs" : 0
},
{
"_id" : 4,
"name" :"mongohost2:28032",
"health" : 1,
"state" : 7,
"stateStr" :"ARBITER",
"uptime" :4074,
"lastHeartbeat" : ISODate("2015-07-17T11:09:51Z"),
"lastHeartbeatRecv" :ISODate("2015-07-17T11:09:51Z"),
"pingMs" : 0
},
{
"_id" : 5,
"name" :"mongohost3:28032",
"health" : 1,
"state" : 7,
"stateStr" :"ARBITER",
"uptime" :4074,
"lastHeartbeat" :ISODate("2015-07-17T11:09:51Z"),
"lastHeartbeatRecv" :ISODate("2015-07-17T11:09:51Z"),
"pingMs" : 0
}
],
"ok" : 1
}
shard2:SECONDARY>
优化
优化内存策略
echo 0 > /proc/sys/vm/zone_reclaim_mode
vi /proc/sys/vm/zone_reclaim_mode
sysctl -w vm.zone_reclaim_mode=0
常用命令行
mongod :MonoDB 服务端
mongo ：MongoDB 客户端
mongoimport： 数据库导入工具
mongoexport ： 数据导出工具
mongodump ： 数据库备份工具
mongorestore： 数据库恢复工具
mongofiles ： GridFS管理工具，可实现对二进制文件的存取
mongostat ： 性能分析工具，类似与vmstat
mongotop ：性能分析工具，查看mongodb实例读写时间花费
mongoperf ：性能分析工具 ，磁盘性能检查工具
mongos ： MongoDB集群工具
转载于:https://blog.51cto.com/hexudong/1677040