Introduction to Site-to-Site Communications in SMS 2003 and Configuration Manager 2007:-
In a Microsoft Systems Management Server 2003 or Microsoft System Center Configuration Manager 2007 site hierarchy, each site must be able to communicate with its parent site and all of its child sites. This communications capability is one of the features that enables the product to scale to levels necessary to make it an Enterprise-wide solution.
Communication between sites is accomplished by using the Server Message Block (SMB) protocol (TCP/IP port 445) and is independent of the SMS 2003 security mode (Standard Security mode or Advanced Security mode) or the Configuration Manager 2007 site mode (native mode or mixed mode).
For sites to communicate, they must have a connectivity system (LAN protocols, RAS, or SNA Server) installed and configured according to the connectivity system’s product documentation on all site servers in the SMS 2003 or Configuration Manager 2007 hierarchy. Then, for each site in the hierarchy, you must configure the site-to-site communications by creating and configuring the required addresses and senders.
SMS 2003 and Configuration Manager 2007 sites communicate using package routing. During package routing, communications are passed up and down a hierarchy from site to site. This means that a site needs addresses only for its parent site and child sites, but not for other upper-level, lower-level, or sibling sites.
Most of the individual SMS 2003 and Configuration Manager components use site-to-site communications to replicate data objects between sites. At times it is necessary to know at their deepest levels the functional details of the components and the data flow paths involved in site-to-site communications in order to effectively troubleshoot other components. This document explains these details and illustrates the data flow paths of these components. This information can help you to resolve issues more quickly and efficiently.
We take a detailed look at the individual components associated with this site-to-site communication.
The information in this section is applicable to both SMS 2003 and Configuration Manager 2007 because the functionality and design is the same in both products.
Address: Contains configured destination site code, site server name, site address account, schedule, and rate limits.
Active components with log files:-
REPLICATION_MANAGER (replmgr) is an SMSEXECUTIVE thread component that processes both incoming and outgoing objects that are replicated to or from a different site within the hierarchy. Replication Manager Packages outbound objects into replication files and hands them off to Scheduler as mini-jobs to be sent to the destination sites by a sender. Replication Manager also analyzes incoming replication files and moves the files to the targeted inbox folder on the destination site server.
An SMS 2003 or Configuration Manager 2007 component requests replication by dropping a request into the Replication Manager inbox. Replication Manager processes the request and places the outbound replication object in one of the High, Normal or Low directories under \inboxes\replmgr.box\outbound\, dependant on the replication priority of the request. This causes a directory change notification. All three of these directories are scanned in descending order of priority. All outbound files for a given site are moved to ..\Inboxes\replmgr.box\process\<SITECODE> directory. Note that if Replication Manager receives additional files that have the same replication priority and are destined for the same site before objects within an existing scheduled replication job are sent, the newly received files will also be moved into the \replmgr.box\process\<SITECODE> directory. Adding newly received files to the folder until immediately prior to scheduling results in more efficient replication as fewer replication jobs are required. Replication Manager continues to scan until one of the following conditions is met:
- No more files exist
- Elapsed time is longer than two polling intervals (polling interval is 5 minutes)
- There has been no activity for 5 seconds
Replmgr then evaluates the scanned files. It then either creates a replication package and Scheduler mini-job or leaves them in the replmgr.box\Process\sitecode folder to be sent later. The replication package is created in inboxes\replmgr.box\ready\*.sitecode folder; where * is a random name given to the folder and sitecode is the destination sitecode. One such folder is created per replication job. The mini-job file (#.job) is created in \Inboxes\Schedule.box\. Of particular note is that if a replication job started 6 hours ago (or longer), Replication Manager will create an additional compressed replication package and minijob destined for the applicable site.
Note that after a job has been created, it cannot be modified. New replication data can and will be added to the queue (Replmgr.box\Process\Sitecode). The next job to be created will schedule all data within the folder for sending.
Immediately prior to scheduling, replmgr creates a compressed package consisting of all the objects that await scheduled replication to the destination site. Replmgr generates a transaction ID (if required) for the package, moves the outbound replication files to replmgr.box\ready, and then creates a mini-job for the scheduler to schedule the package.
Some SMS 2003 and Configuration Manager 2007 components use transaction-based processing for replication jobs. The individual components themselves request this feature at the time they generate a replication request for Replication Manager. Replication Manager in turn generates a unique transaction ID for each replication job in such instances. For each transaction-based replication request received by Replication Manager, the site transaction ID maintained in site server’s registry is incremented by one. This incremented transaction ID is imbedded in the replication file transmitted to the destination site. This affects “transaction-based” replication objects (site control files, collections, packages, and status summarizers). Transaction-based replication objects are named differently by Replication Manager. They are given a three-letter filename extension of .RPT. Non-transaction based replication objects processed by Replmgr are given a three-letter filename extension of .RPL.
Each site server maintains the following:
It’s own unique Transaction ID in the following registry key:
This Transaction ID is used only for outbound replication traffic and is incremented by one for each new outbound replication request that requires a transaction ID.
Transaction IDs from other sites (either parent or child) that it has received transaction enabled replication objects from. The destination/receiving site writes transaction ID numbers contained within replication jobs received from source/sending sites to a history file at \SMS\Inboxes\Replmgr.box\history\sitecode.trs.
Inbound replication jobs that use transactions must bear a transaction ID that is greater than the existing number for that object type that is contained in the sitecode.trs present on the destination site. If this condition is met, the transaction is considered valid and the inbound replication objects are passed to the destination component/inbox. If this condition is not met, the transaction is considered invalid and the replication objects are discarded by Replication Manager.
Each time an inbound replication file with a higher transaction ID number than the ID number in the history is processed, a history file reset is performed. This updates the history file with the new highest number processed for that object type. Each receiving site creates a text history (.trs) file for each site (child or parent) that sends transaction-based replication jobs to it. This process doesn’t ensure delivery of each replication object. Instead, it only guarantees that receiving site replication components won’t process older versions of a replication object that has already been processed by the site.
SMS_SCHEDULER is an SMSEXECUTIVE thread component that manages data transfer between sites and it is based on user-defined addresses and sending schedules. SMS_SCHEDULER receives replication job requests from Replication Manager, Distribution Manager, and Hierarchy Manager and wakes up senders to transfer the files. Scheduler also keeps track of the job status until sending is finished. After the job is finished, Scheduler updates the status of the job to complete and then deletes the job.
The Scheduler cycle consists of the following actions. Note that scheduler cycles through this sequence of actions while the scheduler thread is active.
Processing jobs – Scheduler processes the job requests created in inboxes\replmgr.box\ready by Replication Manager, Distribution Manager, and Hierarchy Manager. This processing includes creating a single compressed package file that contains all the files located in the inboxes\replmgr.box\ready folder that are to be replicated. An instruction file is also created that instructs the Despooler component what specific actions to take on the replicated package file. The instruction and package files are created in schedule.box\tosend.
Processing Routing Requests- Scheduler processes routing requests only on middle tier sites. These requests are used to send packages from the middle tier sites to 3rd and lower tier sites. This is done so that the parent site is required to send the package only to the middle tier site. Subsequent sending to 3rd tier sites is accomplished by the middle tier site. Routing requests exist only at the middle tier sites.
Updating Send Request List – Scheduler updates the send request list after performing the processing specified in the first two steps above.
Building Outbox Send Request Lists – Scheduler then updates the send request lists for the jobs waiting to be actively scheduled and sent.
Checking Status of All Send Requests – Then scheduler loops through and checks the status (progress) of all send requests. During this time Scheduler may attempt to suspend in-progress sender transmissions because of bandwidth throttling, address closure, etc. If there are send requests that haven’t been updated for a long time, Scheduler tries to restart them.
Scheduling All Unscheduled Send Requests – Scheduler performs scheduling of all send requests that are currently not scheduled for sending.
Cleaning Up Routing Packages – Scheduler deletes routing requests older than 25 hours that do not have send requests still pending.
SMS_LAN_SENDER is an SMSEXECUTIVE thread component that uses an existing connectivity system to facilitate site-to-site communication and to send instructions data objects among sites. LAN sender manages connections, ensures the integrity of transferred data, recovers from errors, and closes connections when they are no longer needed. It also honors the bandwidth controls defined by the address for the destination site.
COURIER_SENDER is an SMSEXECUTIVE thread component that writes site data on physical media such as compact discs, floppy disks, or tapes to be sent to other sites when the only available bandwidth is very limited. On the sending site, SMS Administrators must create a courier sender address so that Scheduler will create the courier sender outbox. Administrators can open the Courier Sender program on the source site to create the outgoing parcel file. The administrator can copy the parcel file to removable storage media. On the receiving site, the administrator opens the courier sender program and imports the parcel file from the media and unpacks the parcel files and sends them over to Despooler for processing.
RAS SENDERS include the following:
Asynchronous line – Use Asynchronous RAS Sender for RAS communications over an asynchronous line.
ISDN line – Uses ISDN RAS Sender for RAS communications over an ISDN line.
25 line – Uses X25 RAS Sender for RAS communications over an X.25 line.
SNA – Uses Systems Network Architecture (SNA) RAS Sender in RAS communications over an SNA link.
NOTE: SMS 2003 allows you to install the same sender type on multiple servers within a single site. Configuration Manager 2007does not allow this configuration.
DESPOOLER is an SMSEXECUTIVE thread component that processes instruction files replicated by the Sender component on the source site and it performs the actions specified by these instruction files. There are several different action types that can be specified by the instruction file. Instruction files created by the Scheduler component on the source site are always sent AFTER the accompanying package file. This ensures that Despooler does not try to initiate action on a package file that has not yet replicated or is only partially replicated to the destination site.
How Despooler processes instruction files:
If Scheduler on the source site creates both a package file and instruction file:
- a) Sender on the sending site writes. PCK file to Despoolr.box\receive
- b) Sender begins writing the instruction file as a .TMP file to Despoolr.box\receive and renames the *.TMP to *.SNI when done. The PCK and SNI files have the same name, but different filename extensions.
- c) Despooler renames PCK file to DS_*.PKG
- d) Despooler renames SNI file to DS_*.IST. The PKG and IST files will also have the same names but different filename extensions.
- e) Despooler then takes the IST file name, gives it a PKG extension (only in a variable, not the actual file) and then tries to open the PKG file with this name. This is done immediately after it logs “Begin to calculate…” in its log file.
If there is no package file, only an instruction:
- a) Sender writes TMP files to Despoolr.box\receive and renames it to SNI when complete.
- b) Despooler sees SNI file and checks for a matching PCK file. If not found, it renames the SNI file to DS_*.NIL. (This would be the case, for example, if a resynchronization instruction came from a parent site).
- c) This instruction type doesn’t have a signature, so you won’t see the “Begin to calculate …” entry in schedule.log. The instruction is run and the NIL file is deleted.
There’s is also a scenario where Despooler receives an INS file instead of an SNI file, but this is handled the same way as “1. If Scheduler on the source site creates both a package file and instruction file:”
Instruction File Action Types:-
Typically, most instruction file action results in Despooler transferring the accompanying package file to replmgr\inbox. Exceptions to this behavior include the following:
If the instruction file contains a software distribution package routing instruction, Despooler uses Scheduler to create a send request for the compressed package file or package delta file to be scheduled for sending to the destination site. This send request functions on middle-tier sites to send existing packages from that middle-tier site to child sites.
If the package file contains an inventory resync instruction, Despooler adds a resynchronization request file (*.cfg) in the \\<ServerName>\SMS_<SiteCode>\Clidata.src directory.
If the replicated package file contains inventory .mif files from child sites, Despooler copies them to dataldr.box.
Site Control files (*.ct0) replicated to child sites by HIERARCHY_MANAGER at a parent site are written to the child site’s \Despoolr\Receive folder. Despooler at the child site bypasses Replication Manager and moves the new site control file to Hman.box.
If the replicated package files contains a compressed package source file, Despooler itself updates the master compressed copy of the package on the site by merging the changes in the package content (or copies the .pck to the SMSPkg folder if this is an initial package replication ) instead of passing the package to Distribution Manager. Despooler then updates the database with the package information.
Replication Component Flow:-
Most replication flow is from the requesting component thread to Replication Manager to Scheduler to Sender on the outbound site to Despooler, Replication Manager, and then the applicable thread component on the receiving site. However, two SMSExecutive thread components bypass Replication Manager on the outbound site in specific situations and directly contact the Scheduler thread to create a .job file and request that the Scheduler component schedule outbound replication. These situations are as follows:
Secondary site installation initiated at parent site – If CD installation is not selected, Hierarchy Manager on the parent site starts a processing thread to compress the installation files into the \inboxes\hman.box\Sitepkg.p* on the parent server. Hierarchy Manager then creates a mini-job for Scheduler (bypasses replmgr) to replicate this site installation package.
Distribution Manager package source (.pck) replication – Distribution Manager bypasses Replication Manager on the source site and creates a job file (*.job) when replicating compressed package source files (*.pck). Scheduler processes the job request by waking up the appropriate sender to transfer the compressed package source (*.pck) file. Note that Distribution Manager package source replication also bypasses Replication Manager on the receiving site. Despooler updates the master compressed copy of the package on the destination site by merging the changes in the package content (or copies the .pck to the SMSPkg folder if this is an initial package replication. Despooler then updates the database with the package information. Only then does Distribution Manager act on the package and perform the package action. The section above that details Despooler functionality also refers to four other circumstances in which Despooler on the receiving site bypasses Replication Manager on the destination site and copies replicated files to the appropriate SMSEXECUTIVE thread component inbox.
This example assumes replication processes that do not bypass Replication Manager on neither the source (sending) site nor the destination (receiving) site.
Normal intrasite components do their assigned work. When assigned work includes data that must be replicated to a different site, the following actions occur:
A replication request is made to Replication Manager. The thread component places a replication file (*.RPL or *.RPT) in \inboxes\replmgr.box\outbound\priority). Replication Manager creates a replication job or mini-job in inboxes\replmgr.box\ready. The mini-job is copied to schedule.box.
Scheduler creates a send request (.SRQ file) in the Schedule.box\Requests folder, an instruction file (.INS file) and a package file (.P* file) in schedule.box\tosend folder. If the package is destined for a grandchild site, we also create and transmit a routing instruction file (.RPG). After scheduling, the appropriate sender is selected and the send request (.SRQ) file is moved to the \inboxes\schedule.box\outboxes\sender\*.srq where sender is the name of the sender being used. The Sender component renames \inboxes\schedule.box\outboxes\sender\*.SRQ to \inboxes\schedule.box\outboxes\sender\*.SRS and begins sending the data. Sender updates the .SRS file during transmission. When sending is complete, Sender creates a .DWS file in schedule.box. Scheduler’s cleanup cycle deletes the .SRS and . DWS file.
Despooler receives the data in Despool.box\receive. Despooler decompresses the package and transfers the accompanying package file to replmgr.box\incoming. Replication Manager determines the target component and transfers the replication objects to the appropriate component’s inbox folder.