您的位置: 首页  >  文章  >  mongodb sharding分布式

mongodb sharding分布式

分类: 文章 • 2024-02-02 08:50:58

mongodb分布需要以下三种类型服务

1.Query Routers:用来直接面对用户，路由数据写入路径

2.Config servers:用来存储分布式的元数据，即具体数据在各服务上的分布情况，至少三个Config server

3.Shards:存储数据，其中Shard最好是Replica Set，并且至少两个Shard

mongodb sharding分布式
1.首先要先部署好shard集群

配置文件示例如下：



systemLog:
 quiet: true
 destination: file
 path: "/opt/local/mongodb-conf/logs_pisa_fls/system.log"
 logAppend: true
processManagement:
 fork: true
net:
 port: 8000
 bindIp: 10.130.161.98
storage:
 dbPath: "/opt/local/mongodb-conf/data_pisa_fls/"
 journal:
 enabled: true
 smallFiles: true
replication:
 replSetName: "pisa_fls_1"
sharding:
 clusterRole: shardsvr


此处由于是在一台虚拟机上进行的测试，所以将smallFiles选项设为true，否则每个mongodb服务都装初始化一个占用物理磁盘5%空间的oplog，空间会不够用


2.配置Config servers


配置文件示例如下




systemLog:
 quiet: true
 destination: file
 path: "/opt/local/mongodb-conf-2/logs_pisa_fls_2/system.log"
 logAppend: true
processManagement:
 fork: true
net:
 port: 8006
 bindIp: 10.130.161.98
storage:
 dbPath: "/opt/local/mongodb-conf-2/data_pisa_fls_2/"
 journal:
 enabled: true
sharding:
 clusterRole: configsvr


3.配置Query Routers


Query Routers一定是在配好Config server并且已运行的情况下再进行配置和启动


配置示例如下




systemLog:
 quiet: true
 destination: file
 path: "/opt/local/mongodb-router-conf/logs_pisa_fls_router/system.log"
 logAppend: true
processManagement:
 fork: true
net:
 port: 8009
 bindIp: 10.130.161.98
sharding:
 chunkSize: 1024
 configDB: 10.130.161.98:8006,10.130.161.98:8007,10.130.161.98:8008


其中 configDB项为所需的Config server，router不需要数据存储项，否则会报错


4.启动服务


Config servers和Shards用mongod启动即可，例：




./mongod --config /opt/local/mongodb-conf/mongodb_pisa_fls.conf 


Query Routers需要用mongos启动，例：




./mongos --config /opt/local/mongodb-router-conf/mongodb_pisa_fls_router.conf


5.关联集群


首先shell登录Query Routers




./mongo --host 10.130.161.98 --port 8009


使用sh.addShard([replica-set-name]/[hostname]:port)添加集群






mongos> sh.addShard("pisa_fls_1/10.130.161.98:8000")


{ "shardAdded" : "pisa_fls_1", "ok" : 1 }








mongos> sh.addShard("pisa_fls_2/10.130.161.98:8003")


{ "shardAdded" : "pisa_fls_2", "ok" : 1 }




设置数据库为可分片






mongos> sh.enableSharding("pisa_fls")


{ "ok" : 1 }




创建要分片的集合






mongos> db.createCollection("geographic")


{


 "ok" : 1,


 "$gleStats" : {


 "lastOpTime" : Timestamp(1411103931, 1),


 "electionId" : ObjectId("541ba83559d3eec53f313ec7")


 }


}




创建集合索引






mongos> db.geographic.createIndex({loc:"2dsphere"})


{


 "raw" : {


 "pisa_fls_1/10.130.161.98:8000,10.130.161.98:8001,10.130.161.98:8002" : {


 "createdCollectionAutomatically" : false,


 "numIndexesBefore" : 1,


 "numIndexesAfter" : 2,


 "ok" : 1


 }


 },


 "ok" : 1


}




设置集合为分片模式sh.shardCollection("db name:collection name", shardKey)




mongos> sh.shardCollection("pisa_fls.geographic", {"_id":"hashed"})


显示当前分布式状态






mongos> sh.status()


--- Sharding Status --- 


  sharding version: {


 "_id" : 1,


 "version" : 4,


 "minCompatibleVersion" : 4,


 "currentVersion" : 5,


 "clusterId" : ObjectId("541ba6bbe965332128382725")


}


  shards:


 {  "_id" : "pisa_fls_1",  "host" : "pisa_fls_1/10.130.161.98:8000,10.130.161.98:8001,10.130.161.98:8002" }


 {  "_id" : "pisa_fls_2",  "host" : "pisa_fls_2/10.130.161.98:8003,10.130.161.98:8004,10.130.161.98:8005" }


  databases:


 {  "_id" : "admin",  "partitioned" : false,  "primary" : "config" }


 {  "_id" : "test",  "partitioned" : false,  "primary" : "pisa_fls_1" }


 {  "_id" : "pisa_fls",  "partitioned" : true,  "primary" : "pisa_fls_1" }


 pisa_fls.geographic


 shard key: { "_id" : "hashed" }


 chunks:


 pisa_fls_1 2


 pisa_fls_2 2


 { "_id" : { "$minKey" : 1 } } -->> { "_id" : NumberLong("-4611686018427387902") } on : pisa_fls_1 Timestamp(2, 2) 


 { "_id" : NumberLong("-4611686018427387902") } -->> { "_id" : NumberLong(0) } on : pisa_fls_1 Timestamp(2, 3) 


 { "_id" : NumberLong(0) } -->> { "_id" : NumberLong("4611686018427387902") } on : pisa_fls_2 Timestamp(2, 4) 


 { "_id" : NumberLong("4611686018427387902") } -->> { "_id" : { "$maxKey" : 1 } } on : pisa_fls_2 Timestamp(2, 5) 




列出各集群状态




mongos> use admin


switched to db admin


mongos> db.runCommand({listShards:1})


{


 "shards" : [


 {


 "_id" : "pisa_fls_1",


 "host" : "pisa_fls_1/10.130.161.98:8000,10.130.161.98:8001,10.130.161.98:8002"


 },


 {


 "_id" : "pisa_fls_2",


 "host" : "pisa_fls_2/10.130.161.98:8003,10.130.161.98:8004,10.130.161.98:8005"


 }


 ],


 "ok" : 1


}





查看集合详细分布情况




mongos> db.geographic.stats()
{
 "sharded" : true,
 "systemFlags" : 1,
 "userFlags" : 1,
 "ns" : "pisa_fls.geographic",
 "count" : 20,
 "numExtents" : 2,
 "size" : 4800,
 "storageSize" : 16384,
 "totalIndexSize" : 49056,
 "indexSizes" : {
 "_id_" : 16352,
 "_id_hashed" : 16352,
 "loc_2dsphere" : 16352
 },
 "avgObjSize" : 240,
 "nindexes" : 3,
 "nchunks" : 4,
 "shards" : {
 "pisa_fls_1" : {
 "ns" : "pisa_fls.geographic",
 "count" : 7,
 "size" : 1680,
 "avgObjSize" : 240,
 "storageSize" : 8192,
 "numExtents" : 1,
 "nindexes" : 3,
 "lastExtentSize" : 8192,
 "paddingFactor" : 1,
 "systemFlags" : 1,
 "userFlags" : 1,
 "totalIndexSize" : 24528,
 "indexSizes" : {
 "_id_" : 8176,
 "loc_2dsphere" : 8176,
 "_id_hashed" : 8176
 },
 "ok" : 1
 },
 "pisa_fls_2" : {
 "ns" : "pisa_fls.geographic",
 "count" : 13,
 "size" : 3120,
 "avgObjSize" : 240,
 "storageSize" : 8192,
 "numExtents" : 1,
 "nindexes" : 3,
 "lastExtentSize" : 8192,
 "paddingFactor" : 1,
 "systemFlags" : 1,
 "userFlags" : 1,
 "totalIndexSize" : 24528,
 "indexSizes" : {
 "_id_" : 8176,
 "loc_2dsphere" : 8176,
 "_id_hashed" : 8176
 },
 "ok" : 1
 }
 },
 "ok" : 1
}


															

																					

								

									相关推荐