xstream-guide_19c 文档笔记
好长的文档,周末看了100多页,记录些概念
XStream 由DB组件和API接口组成,能让客户端应用接收源DB数据变化并发送至目标端,目标端可以是非oracle,例如sqlserver/pg/文件系统/第三方软件应用等。
XStream是一个程序接口,允许客户端应用连接到oracle数据库并且之间访问db capture process或apply process
XStream 有两个主要部分:XStream Out 和 XStream In
XStream Out在源端,从redolog中接收源端的DML,DDL语句,并将变化情况通过接口发送至客户端应用,也可以选择将变化情况存储到memory cache或指定文件中
XStream In在目标端,apply从客户端应用接收到的数据变化情况,使目标库同步源库数据。
实现基础:
- 数据变化情况使用 logical change record (LCR) 格式存储
一条dml语句可能会操作多行,因此可能会对应产生多条row LCR记录;如果是ddl语句,则会对应产生一条DDL LCR记录.
- 规则和规则集用于控制XStream行为,规则包括inclusion and exclusion rules,支持在DB级、schema级、表级、行/列级设置。
- 基于规则的数据变化会被捕获
-
支持DB中众多数据类型,包括LOBs,
LONG
,LONG
RAW
, andXMLType
- 支持配置一对多、多对一等情况
- 支持并行apply、sql生成、冲突检测和解决、错误处理、使用apply handlers定制apply
使用前提
有oracle DB相关知识
有分布式数据库相关知识
有sql和plsql知识
有应用编程知识,需要使用OCI或JAVA API创建客户端应用并与XStream交互
XStream安全模型
XStream支持两类安全模式:XStream Trusted User Model(可管理XStream配置,有更多DB权限,可通过DBA_视图监控XStream运行情况,更易实现)、XStream Untrusted User Model(只能管理XStream配置,可通过ALL_视图监控XStream运行情况)
- XStream相关概念
主要有以下几项
- Logical Change Records (LCRs)
- Rules and Rule Sets
- Rule-Based Transformations
- XStream and the Oracle Replication Performance Advisor
- XStream and SQL Generation
Logical Change Records (LCRs)
LCR是一种有特定格式的用于模式DB变化情况的消息(An LCR is a message with a specific format that describes a database change.),它是XStream中描述数据变化的基本单位( an LCR is the basic unit of information that describes a database change)
在XStream Out 配置中,capture process 会捕获LCR信息并将它们发送到 outbound server, outbound server再将LCR发送到客户端应用
在XStream In 配置中,客户端应用将LCR发送到inbound server,inbound server可以将LCR发送到DB直接进行apply,也可以对LCR进行客制化处理
LCR有三种类型: row LCRs, DDL LCRs, and sequence LCRs(包含序列值的row LCR)
LCR都有LCRID,用于标志和排序LCR,通过它可以确定哪些LCR和事务已被接收和应用
row LCR 的类型为LCR$_ROW_RECORD,有以下属性:
Attribute |
Description |
source_database_name |
The name of the source database where the row change occurred. If the LCRs originated in a multitenant container database (CDB), then this attribute specifies the global name container where the row change occurred. |
command_type |
The type of DML statement that produced the change, either INSERT, UPDATE, DELETE, LOB ERASE, LOB WRITE, or LOB TRIM. |
object_owner |
The schema name that contains the table with the changed row. |
object_name |
The name of the table that contains the changed row. |
tag |
A raw tag that you can use to track the LCR. |
transaction_id |
The identifier of the transaction in which the DML statement was run. |
scn |
The system change number (SCN) at the time when the change was made. |
old_values |
The old column values related to the change. These are the column values for the row before the DML change. If the type of the DML statement is UPDATE or DELETE, then these old values include some or all of the columns in the changed row before the DML statement. If the type of the DML statement is INSERT, then there are no old values. For UPDATE and DELETE statements, row LCRs created by a capture process can include some or all of the old column values in the row. |
new_values |
The new column values related to the change. These are the column values for the row after the DML change. If the type of the DML statement is UPDATE or INSERT, then these new values include some or all of the columns in the changed row after the DML statement. If the type of the DML statement is DELETE, then there are no new values. For UPDATE and INSERT statements, row LCRs created by a capture process can include some or all of the new column values in the row. |
position |
A unique identifier of RAW data type for each LCR. The position is strictly increasing within a transaction and across transactions. LCR position is commonly used in XStream configurations. |
root_name |
If the LCR originated in a CDB, then this attribute specifies the global name of the root in the CDB. If the LCR originated in a non-CDB, then this attribute is the same as the source_database_name attribute. |
额外属性
Attribute |
Description |
commit_scn |
The commit system change number (SCN) of the transaction to which the LCR belongs. |
commit_scn_from_position |
The commit system change number (SCN) of a transaction determined by the input position, which is generated by an XStream outbound server. |
commit_time |
The commit time of the transaction to which the LCR belongs. |
compatible |
The minimal database compatibility required to support the LCR. |
instance_number |
The instance number of the database instance that made the change that is encapsulated in the LCR. Typically, the instance number is relevant in an Oracle Real Application Clusters (Oracle RAC) configuration. |
lob_information |
The LOB information for the column, such as NOT_A_LOB or LOB_CHUNK. |
lob_offset |
The LOB offset for the specified column in the number of characters for CLOB columns and the number of bytes for BLOB columns. |
lob_operation_size |
The operation size for the LOB column in the number of characters for CLOB columns and the number of bytes for BLOB columns. |
long_information |
The LONG information for the column, such as NOT_A_LONG or LONG_CHUNK. |
row_text |
The SQL statement for the change that is encapsulated in the row LCR. |
scn_from_position |
The SCN of the LCR. |
source_time |
The time when the change in an LCR captured by a capture process was generated in the redo log of the source database, or the time when a persistent LCR was created. |
xml_information |
The XML information for the column, such as NOT_XML, XML_DOC, or XML_DIFF. |
DDL LCR 的类型为LCR$_DDL_RECORD,有以下属性:
Table 2-3 Attributes Present in All DDL LCRs
Attribute |
Description |
source_database_name |
The name of the source database where the DDL change occurred. If the LCRs originated in a CDB, then this attribute specifies the global name of the container where the DDL change occurred. |
command_type |
The type of DDL statement that produced the change, for example ALTER TABLE or CREATE INDEX. |
object_owner |
The schema name of the user who owns the database object on which the DDL statement was run. |
object_name |
The name of the database object on which the DDL statement was run. |
object_type |
The type of database object on which the DDL statement was run, for example TABLE or PACKAGE. |
ddl_text |
The text of the DDL statement. |
logon_user |
The logon user, which is the user whose session executed the DDL statement. |
current_schema |
The schema that is used if no schema is specified for an object in the DDL text. |
base_table_owner |
The base table owner. If the DDL statement is dependent on a table, then the base table owner is the owner of the table on which it is dependent. |
base_table_name |
The base table name. If the DDL statement is dependent on a table, then the base table name is the name of the table on which it is dependent. |
tag |
A raw tag that you can use to track the LCR. |
transaction_id |
The identifier of the transaction in which the DDL statement was run. |
scn |
The system change number (SCN) at the time when the change was made. |
position |
A unique identifier of RAW data type for each LCR. The position is strictly increasing within a transaction and across transactions. LCR position is commonly used in XStream configurations. |
edition_name |
The name of the edition in which the DDL statement was executed. |
root_name |
If the LCR originated in a CDB, then this attribute specifies the global name of the root in the CDB. If the LCR originated in a non-CDB, then this attribute is the same as the source_database_name attribute. |
额外属性
Table 2-4 Additional Attributes in DDL LCRs Captured by a Capture Process
Attribute |
Description |
commit_scn |
The commit system change number (SCN) of the transaction to which the LCR belongs. |
commit_scn_from_position |
The commit SCN of a transaction determined by the input position, which is generated by an XStream outbound server. |
commit_time |
The commit time of the transaction to which the LCR belongs. |
compatible |
The minimal database compatibility required to support the LCR. |
instance_number |
The instance number of the database instance that made the change that is encapsulated in the LCR. Typically, the instance number is relevant in an Oracle Real Application Clusters (Oracle RAC) configuration. |
scn_from_position |
The SCN of the LCR. |
source_time |
The time when the change in an LCR captured by a capture process was generated in the redo log of the source database, or the time when a persistent LCR was created. |
Extra Information in Row LCRs and DDL LCRs
In addition to the information discussed in the previous sections, row LCRs and DDL LCRs optionally can include extra information (or LCR attributes).
The extra attributes in LCRs are described in the following table.
Table 2-5 Extra Attributes in LCRs
Attribute |
Description |
|
The rowid of the row changed in a row LCR. This attribute is not included in DDL LCRs or row LCRs for index-organized tables. |
|
The serial number of the session that performed the change captured in the LCR. |
|
The identifier of the session that performed the change captured in the LCR. |
|
The thread number of the instance in which the change captured in the LCR was performed. Typically, the thread number is relevant only in an Oracle Real Application Clusters (Oracle RAC) environment. |
|
The name of the transaction that includes the LCR. |
|
The name of the current user who performed the change captured in the LCR. |
You can use the INCLUDE_EXTRA_ATTRIBUTE
procedure in the DBMS_CAPTURE_ADM
package to instruct a capture process to capture one or more extra attributes.
Sequence LCR是包含序列值的row LCR
可以通过如下方法捕获
DB层:将capture_sequence_nextval
和apply_sequence_nextval
参数设为y
OCI接口:use the OCILCRNew
function and the OCILCRHeaderSet
function with the OCI_ROWLCR_SEQ_LCR
flag.
JAVA接口: use the DefaultRowLCR
constructor and setSequenceLCRFlag
method
如何定位LCR流顺序
每个LCR都有position属性,用于标志该LCR在LCR流和事务中的顺序
position属性有以下特点:
每个LCR的position属性是唯一的
position属性是RAW
data type.
Position随LCR流和事务严格递增
- The position is byte-comparable, and the comparison results for multiple positions determines the ordering of the LCRs in the stream.
- The position of an LCR remains identical when the database, the client application, or an XStream component restarts.
- The position is not affected by any rule changes that might reduce or increase the number of LCRs in the stream.
XStream Out only sends committed data, and XStream In only receives committed data.
LCRID用于在XStream Out中唯一标志LCR的位置,它同样是严格递增的。
从12.2开始,the LCRID is versioned。当创建或新增outbound server时,可以选择使用的版本。要使用version 2,DB版本必须高于12.2。低于12.2版本默认会使用version 1,12.2开始默认使用version 2。如果源库版本为12.2或以上,目标库版本低于源库,可能需要手动设置该参数。
outbound server创建或添加完成后,不能再修改LCRID version,如果要改,只能drop该outbound server重建。如果outbound server正在发送LCR到inbound server,还需要drop该inbound server重建。
相同的DB变化在version 1和version 2中有不同的值,可以使用DBMS_XSTREAM_ADM
包可用于比较不同版本的LCRID(使用COMPARE_POSITION
函数)或将一个版本的LCRID转为另一个版本(使用CONVERT_POSITION
函数)
规则与规则集
Rules and Rule Sets Defined
规则是一个数据库对象,定义客户端执行某种动作的条件。在XStream配置中,规则标志哪些LCRs会被传送到另一个组件
A rule is a database object that enables a client to perform an action when an event occurs and a condition is satisfied. In an XStream configuration, rules identify which LCRs to stream from one component to another.
Capture processes, propagations, outbound servers and inbound servers都可以配置规则,规则适用于dml和ddl语言。
规则集是规则的集合,可以配置positive rule set and a negative rule set ,并且negative rule优先
Rule Sets and XStream Components
当database change符合规则时,XStream component会执行指定的任务
如果redolog中的change符合为capture process设置的规则,它会被捕获,如果不满足则被略过。
如果捕获到的LCR满足为propagations设置的规则,它会被发送,如果不满足则被略过。
如果被发送的LCR满足为outbound servers设置的规则,它会被发送到客户端应用,如果不满足则被略过。
如果inbound servers接收到的LCR满足为其设置的规则,它会被apply,如果不满足则被略过。
如果不设置规则,所有的database changes都会被捕获、发送、apply
System-Created Rules and XStream
A system-created rule is created by the DBMS_XSTREAM_ADM
package.
Table 2-6 XStream System-Created Rule Procedures
Procedure |
Capture Process |
Propagation |
Outbound Server |
Inbound Server |
CREATE_OUTBOUND |
Yes |
No |
Yes |
No |
ADD_OUTBOUND |
No |
No |
Yes |
No |
ALTER_OUTBOUND |
Yes |
No |
Yes |
No |
ADD_GLOBAL_RULES |
Yes |
No |
Yes |
Yes |
ADD_GLOBAL_PROPAGATION_RULES |
No |
Yes |
No |
No |
ADD_SCHEMA_RULES |
Yes |
No |
Yes |
Yes |
ADD_SCHEMA_PROPAGATION_RULES |
No |
Yes |
No |
No |
ADD_GLOBAL_RULES |
Yes |
No |
Yes |
Yes |
ADD_SUBSET_OUTBOUND_RULES |
No |
No |
Yes |
No |
ADD_SUBSET_RULES |
Yes |
No |
Yes |
Yes |
ADD_SUBSET_PROPAGATION_RULES |
No |
Yes |
No |
No |
ADD_TABLE_RULES |
Yes |
No |
Yes |
Yes |
ADD_TABLE_PROPAGATION_RULES |
No |
Yes |
No |
No |
system-created rule可以设置为Subset 级、表级、schema级、global级(对整个DB生效)。Subset 级仅对表中部分行生效,类似where语句,需要开启Supplemental logging才能使用。
多租户环境中的system-created rule
Table 2-7 Key Procedure Parameters for System-Created Rules in a CDB
Parameter |
Description |
source_database |
The global name of the source database. In a CDB, specify the global name of the container to which the rules pertain. The container can be the CDB root, a PDB, an application root, or an application PDB. The following are examples: mycdb.example.com or hrpdb.example.com. |
source_root_name |
The global name of the CDB root in the source CDB. The following are examples: mycdb.example.com. |
source_container_name |
The short name of the source container. The container can be the CDB root, a PDB, an application root, or an application PDB. The following are examples: CDB$ROOT or hrpdb. |
Table 2-8 Local Capture and XStream Out Container Rule Conditions
source_database Parameter Setting |
source_container_name Parameter Setting |
Description |
NULL |
NULL |
XStream Out captures and streams changes made in any container in the local CDB, including the CDB root, all PDBs, all application roots, and all application PDBs. |
non-NULL |
NULL |
XStream Out captures and streams changes made in the specified source container of the local CDB. The source container can be the CDB root, a PDB, an application root, or an application PDB. The DBMS_XSTREAM_ADM procedure queries the CDB_PDBS view and CDB_PROPERTIES view to determine the source_container_name value. |
NULL |
non-NULL |
XStream Out captures and streams changes made in the specified source container of the local CDB. The source container can be the CDB root, a PDB, an application root, or an application PDB. The DBMS_XSTREAM_ADM procedure queries the CDB_PDBS view and CDB_PROPERTIES view to determine the source_database value. |
non-NULL |
non-NULL |
XStream Out captures and streams changes made in the specified source container of the local CDB. The source container can be the CDB root, a PDB, an application root, or an application PDB. If the prefix of the source_database value is different from the source_container_name value, then the resulting rules include a condition for the source_database value, and an internal table maps the source_database value to the source_container_name value. |
Downstream Capture and XStream Out Container Rule Conditions
source_database Parameter Setting |
source_container_name Parameter Setting |
Description |
NULL |
NULL |
XStream Out captures and streams changes made in any container in the remote source CDB, including the CDB root, all PDBs, all application roots, and all application PDBs. |
non-NULL |
NULL |
XStream Out captures and streams changes made in the specified source container of the remote source CDB. The source container can be the CDB root, a PDB, an application root, or an application PDB. The DBMS_XSTREAM_ADM procedure derives the source_container_name value from the prefix of source_database value. |
NULL |
non-NULL |
The DBMS_XSTREAM_ADM procedure raises an error. |
non-NULL |
non-NULL |
XStream Out captures and streams changes made in the specified source container of the remote source CDB. The source container can be the CDB root, a PDB, an application root, or an application PDB. If the prefix of the source_database value is different from the source_container_name value, then the resulting rules include a condition for the source_database value, and an internal table maps the source_database value to the source_container_name value. |
Rule-Based Transformations
rule-based transformation是对符合positive 规则的LCR进行指定的转换。这个转换最好是放在客户端应用处理,如果实在不愿意放,db端也可以对DML LCR进行些简单转换。
Declarative Rule-Based Transformations
You specify (or declare) such a transformation using one of the following procedures in the DBMS_XSTREAM_ADM package:
- ADD_COLUMN either adds or removes a declarative transformation that adds a column to a row LCR.
- DELETE_COLUMN either adds or removes a declarative transformation that deletes a column from a row LCR.
- KEEP_COLUMNS either adds or removes a declarative transformation that keeps a list of columns in a row LCR. The transformation removes columns that are not in the list from the row LCR.
- RENAME_COLUMN either adds or removes a declarative transformation that renames a column in a row LCR.
- RENAME_SCHEMA either adds or removes a declarative transformation that renames the schema in a row LCR.
- RENAME_TABLE either adds or removes a declarative transformation that renames the table in a row LCR.
By default, Oracle Database performs declarative transformations in the following order when the rule evaluates to TRUE:
- Keep columns
- Delete column
- Rename column
- Add column
- Rename table
- Rename schema
In addition to declarative rule-based transformations, a row migration is an internal transformation that takes place when a subset rule evaluates to TRUE.
You can use the DBMS_XSTREAM_ADM.ADD_SUBSET_RULES procedure to add subset rules. If both types of transformations are specified for a single rule, then Oracle Database performs the transformations in the following order when the rule evaluates to TRUE:
- Row migration
- Declarative rule-based transformation
User-Specified Declarative Transformation Ordering
If you do not want to use the default declarative rule-based transformation ordering for a particular rule, then you can specify step numbers for each declarative transformation specified for the rule.
使用 rule-based transformations时需考虑项
只会对符合positive rule的LCR生效
rule-based transformations与使用DBMS_TRANSFORM
包进行转换是不同的
如果在XStream In中有大量row LCRs 需要转换,可以使用DML handlers,当然最好还是放在客户端应用模块进行处理
XStream and the Oracle Replication Performance Advisor
Oracle Replication Performance Advisor 由一系列数据字典视图组成,用于监控XStream性能
Performance Advisor tracks the following types of components in an XStream environment:
- QUEUE
- CAPTURE
- PROPAGATION SENDER
- PROPAGATION RECEIVER
- APPLY
Performance Advisor 能帮忙寻找组件瓶颈并找到最忙的组件,可以通过DBA_STREAMS_TP_PATH_BOTTLENECK
视图ACTION_NAME
列查看,一般最忙的会是客户端应用,对应ACTION_NAME
为
EXTERNAL
XStream Out Apply Subcomponents
There are several XStream Out apply subcomponents types.
The following subcomponent types are possible:
-
PROPAGATION
SENDER+RECEIVER
for sending LCRs from a capture process to an outbound server where the capture process and outbound server are in different databases. -
APPLY
READER
for a reader server.APPLY READER
receives LCRs from the capture process, organizes them into transactions, does dependency calculations, and passes the LCRs to the apply coordinator. -
APPLY
COORDINATOR
for a coordinator process. It takes the transactions from the capture process, uses the dependency information to determine how to schedule the transactions and sends the LCRs to the apply server. -
APPLY
SERVER
for an apply server. It delivers the LCRs to the client application.
XStream In Apply Subcomponents
There are several XStream In apply subcomponents types.
The following subcomponent types are possible:
-
APPLY
READER
for a reader server. It takes the LCRs from client application converts them into transactions, checks the transactional order and does dependency calculations. -
APPLY
COORDINATOR
for a coordinator process. It takes the transactions from the reader server, uses the dependency information to determine how to schedule the transactions and sends the LCRs to the apply server. -
APPLY
SERVER
for an apply server. It applies the LCRs to an apply handler. If the LCR cannot be applied, it is placed into an error queue.
Performance Advisor可以跟踪组件级统计信息
The Performance Advisor tracks the following component-level statistics:
- The MESSAGE APPLY RATE is the average number of LCRs applied each second by the apply process, outbound server, or inbound server.
- The TRANSACTION APPLY RATE is the average number of transactions applied by the apply process, outbound server, or inbound server each second. Transactions typically include multiple LCRs.
Performance Advisor还能跟踪 LATENCY
component-level statistics
LATENCY is defined in the following ways:
- For apply processes, the LATENCY is the amount of time between when the LCR was created at a source database and when the LCR was applied by the apply process at the destination database.
- For outbound servers, the apply LATENCY is amount of time between when the LCR was created at a source database and when the LCR was sent to the XStream client application.
- For inbound servers, the apply LATENCY is amount of time between when the LCR was created by the XStream client application and when the LCR was applied by the apply process.
You can collect XStream statistics with the UTL_RPADV
package
Run the utlrpadv.sql
script in the rdbms/admin directory in ORACLE_HOME
to load the UTL_RPADV
package
exec UTL_RPADV.COLLECT_STATS
exec UTL_RPADV.START_MONITORING
The SHOW_STATS
procedure in the UTL_RPADV
package displays the statistics that the Performance Advisor gathered and stored.
Use the path_stat_table
parameter to specify the table that contains the statistics.
SELECT SHOW_STATS_TABLE FROM STREAMS$_PA_MONITORING;
或
SET SERVEROUTPUT ON SIZE 50000
BEGIN
UTL_RPADV.SHOW_STATS(
path_stat_table => 'STREAMS$_PA_SHOW_PATH_STAT');
END;
/
The SHOW_STATS_HTML
procedure in the UTL_RPADV
package creates an HTML report that contains the statistics that the Performance Advisor gathered and stored.
XStream and SQL Generation
SQL generation is the ability to generate the SQL statement required to perform the change encapsulated in a row LCR.
XStream outbound servers and XStream inbound servers can use SQL generation to generate the SQL statement necessary to perform the insert, update, or delete operation in a row LCR.
SQL generation能利用row LCR反向解析出sql语句,可以为 inline values格式和带绑定变量的格式,支持大部分数据类型
You can use the following interfaces to perform SQL generation:
-
The PL/SQL interface, which uses the
GET_ROW_TEXT
andGET_WHERE_CLAUSE
member procedures for row LCRs - The OCI for XStream
- The Java interface for XStream
Part II XStream Out
主要有以下五部分
- XStream Out Concepts
Become familiar with concepts related to XStream Out. - Configuring XStream Out
You can configure the Oracle Database components that are used by XStream Out. - Managing XStream Out
You can manage XStream Out components and their rules. - Monitoring XStream Out
You can monitor an XStream Out configuration. - Troubleshooting XStream Out
You can diagnose and correct problems with an XStream Out configuration.
XStream Out Concepts
XStream Out can capture transactions from the redo log of an Oracle database and send them efficiently to a client application.
Capture Processes
A capture process is an optional Oracle background process that scans the database redo log to capture DML and DDL changes made to database objects.
When a capture process captures a database change, it converts it into a specific message format called a logical change record (LCR). In an XStream Out configuration, the capture process sends these LCRs to an outbound server.
可以通过V$XSTREAM_CAPTURE的STATE
列查看其状态。
When a capture process runs on its source database, the capture process is a local capture process.
You can also capture changes for the source database by running the capture process on different server. When a capture process runs on a remote database, the capture process is called a downstream capture process, and the remote database is called the downstream database
3.2.2.1 ID Key LCRs
An ID key LCR is a special type of row LCR. ID key LCRs enable an XStream client application to process changes to rows that include unsupported data types.
XStream Out does not fully support the following data types in row LCRs:
ROWID
- Nested tables
-
The following Oracle-supplied types:
ANYTYPE
,ANYDATASET
, URI types,SDO_TOPO_GEOMETRY
,SDO_GEORASTER
, andExpression
.
Local Capture and Downstream Capture
You can configure a capture process to run locally on a source database or remotely on a downstream database.
Local Capture
Local capture means that a capture process runs on the source database.
Configuration and administration of the capture process is simpler than when downstream capture is used.
A local capture process can scan changes in the online redo log before the database writes these changes to an archived redo log file.
Downstream Capture
Downstream capture means that a capture process runs on a database other than the source database.
有两种类型: real-time downstream capture and archived-log downstream capture.由downstream_real_time_mine
参数设置
real-time 类型优点在于实时性高,不需要等db change写入redolog和archivelog即可捕获。
archived-log类型优点在于允许downstream database有多个源库。它需要将归档日志copy到downstream database,可以用自带的DBMS_FILE_TRANSFER
包,也可以直接用ftp传
A real-time downstream capture process and one or more archived-log downstream capture processes can coexist(共存) at a downstream database. With downstream capture, the redo log files of the source database must be available at the downstream database.
The following are the advantages of using downstream capture:
捕获change占用源库资源更少,因为基本是在downstream库中完成的
使得多源库的数据同步更简单
将归档数据copy到downstream库相当于给源库多做了一份归档备份,可用于数据恢复
提高了灵活性和可扩展性
The following are operational requirements for using downstream capture:
- The source database must be running at least Oracle Database 10g Release 2 (10.2).
- The XStream Out downstream capture database must be running Oracle Database 11g Release 2 (11.2.0.3) or later and the source database must be running Oracle Database 10g Release 2 (10.2) or later.
- The operating system on the source and downstream capture sites must be the same, but the operating system release does not need to be the same. In addition, the downstream sites can use a directory structure that is different from the source site.
- The hardware architecture on the source and downstream capture sites must be the same. For example, a downstream capture configuration with a source database on a 64-bit Sun system must have a downstream database that is configured on a 64-bit Sun system. Other hardware elements, such as the number of CPUs, memory size, and storage configuration, can differ in the source and downstream sites.
Capture Process Subcomponents
The capture process subcomponents are a reader server, one or more preparer servers, and a builder server.
A capture process is an optional Oracle background process whose process name is CPnn
, where nn
can include letters and numbers. A capture process captures changes from the redo log by using the infrastructure of LogMiner. XStream configures LogMiner automatically. You can create, alter, start, stop, and drop a capture process, and you can define capture process rules that control which changes a capture process captures.
When capture process parallelism is greater than 0, the capture process consists of the following subcomponents:
- One reader server that reads the redo log and divides the redo log into regions.
- One or more preparer servers that scan the regions defined by the reader server in parallel and perform prefiltering of changes found in the redo log. Prefiltering involves sending partial information about changes, such as schema and object name for a change, to the rules engine for evaluation, and receiving the results of the evaluation. You can control the number of preparer servers using the parallelism capture process parameter.
-
One builder server that merges redo records from the preparer servers. These redo records either evaluated to
TRUE
during partial evaluation or partial evaluation was inconclusive for them. The builder server preserves the system change number (SCN) order of these redo records and passes the merged redo records to the capture process.
The capture process (CPnn
) performs the following actions for each change when it receives merged redo records from the builder server:
- Formats the change into an LCR
- If the partial evaluation performed by a preparer server was inconclusive for the change in the LCR, then sends the LCR to the rules engine for full evaluation
- Receives the results of the full evaluation of the LCR if it was performed
- Discards the LCR if it satisfies the rules in the negative rule set for the capture process or if it does not satisfy the rules in the positive rule set
- Enqueues the LCR into the queue associated with the capture process if the LCR satisfies the rules in the positive rule set for the capture process
Capture Process Checkpoints and XStream Out
A capture process tries to record a checkpoint at regular intervals called checkpoint intervals.
- Required Checkpoint SCN
The system change number (SCN) that corresponds to the lowest checkpoint for which a capture process requires redo data is the required checkpoint SCN. - Maximum Checkpoint SCN
The SCN that corresponds to the last physical checkpoint recorded by a capture process is the maximum checkpoint SCN. - Checkpoint Retention Time
The checkpoint retention time is the amount of time, in number of days, that a capture process retains checkpoints before purging them automatically.
SCN Values Related to a Capture Process
Specific system change number (SCN) values are important for a capture process.
You can query the ALL_CAPTURE
data dictionary view to display these values for one or more capture processes.
- Captured SCN and Applied SCN
The captured SCN is the SCN that corresponds to the most recent change scanned in the redo log by a capture process. The applied SCN for a capture process is the SCN of the most recent LCR processed by the relevant outbound server. - First SCN and Start SCN
The first SCN and start SCN are important for a capture process.
Outbound Servers
An outbound server is an optional Oracle background process that sends database changes to a client application.
Specifically, a client application can attach to an outbound server and extract database changes from LCRs. A client application attaches to the outbound server using OCI or Java interfaces.
A client application can create multiple sessions. Each session can attach to only one outbound server, and each outbound server can serve only one session at a time. However, different client application sessions can connect to different outbound servers or inbound servers.
Change capture can be performed on the same database as the outbound server or on a different database. When change capture is performed on a different database from the one that contains the outbound server, a propagation sends the changes from the change capture database to the outbound server database. Downstream capture is also a supported mode to reduce the load on the source database.
An outbound server consists of a reader server, a coordinator process, and an apply server.
A reader server that receives LCRs from the outbound server's capture process. The reader server then returns the assembled transactions to the coordinator process.视图V$XSTREAM_APPLY_READER
A coordinator process that gets transactions from the reader server and passes them to apply servers. The coordinator process name is APnn
视图
V$XSTREAM_APPLY_COORDINATOR
An apply server that sends LCRs to an XStream client application. The apply server is a process. If the apply server encounters an error, then it then it records information about the error in the ALL_APPLY
view.视图 V$XSTREAM_APPLY_SERVER
The reader server and the apply server process names are ASnn
未完待续
参考