Engineering:Telecommunication transaction processing systems

Telecommunication networks can generate a vast amount of transactions where each transaction contains information about a particular subscriber's activity. Telecommunication network consist of various interacting devices and platforms. Any transaction carried out by a subscriber is often recorded in multiple devices as it passes through the network. Telecommunication organizations generally need to be able to extract transaction information from these various network elements in order to correctly bill subscribers for the usage on the network. Transaction processing system is a subset of information systems, and in the telecommunications industry, forms an integral part of the management information system. TPS can be regarded as the link between the various network elements and platforms and the information management uses to drive the business.

Overview of Transaction Processing Systems within a GSM network.

Call Data Records

Each activity occurring on a specific network element within the telecommunication network, is recorded by the particular platform. All available information about the particular transaction is recorded and encoded into different formats. The recorded transactions, is called Call Data Records(CDR). Various formats and protocols are used to encode these CDRs, some example encoding protocols used includes ASN.1, XML and CSV . Some platform vendors develop their own encoding protocols for security reasons. Encoded CDRs are grouped into batches and periodically moved to locations from where the TPS can collect the CDR batches in order to process it.

Gathering of CDR files

The TPS is configured to periodically check each platform for any new CDR batches becoming available. The TPS uses standard network protocols, including FTP, SFTP and FTPS to transfer the CDR batch file to the TPS. Some platform vendors have developed their own file transfer protocols, in which case, the TPS need to be customized in order to retrieve the batch files from these platforms. The TPS is also responsible for ensuring the integrity of each file transferred, ensuring that no IP network errors render the file corrupt. Checking for duplicate files from the particular platform is also a responsibility of the TPS to ensure that no file is processed more than once, resulting in duplication of CDRs. Once batch files are retrieved from a particular network-element, they are backed up to long-term media. Some governments require that a record of each and every transaction needs to be stored infinitely in its raw (encoded) format. The batch sizes and frequency differ for each network-element and is also directly related to the number of active subscribers on a particular telecommunication network.

Decoding / Enrichment and Loading of CDRs

Once the TPS has successfully retrieved all the CDR batches, its first task is to decode the CDRs into human readable (ASCII) format. This is probably one of the most important functions of the TPS within the telecommunication industry, as any error in the decoding process will result in inaccurate, unreliable information being passed onto downstream processes and ultimately to the reports viewed by the management. The TPS usually consists of standard functionality to decode all standard CDR encoding protocols like XML, CSV and ASN.1. Should a particular platform vendor encode CDR's in non standard protocols, customization on the TPS is required. Vendors are then required to supply detailed CDR specifications to the suppliers of the TPS in order to enhance the TPS to recognize the CDR formats and also detailed explanations of which information is contained within the CDR about a particular subscriber's activity on the network. Upon successful completion of the decoding, the decoded CDR batches are then checked for duplicate records. Duplicate CDRs are discarded and reported on. The TPS administrators are responsible for verifying the CDRs flagged as duplicates, are in fact true duplicates.

Depending on the TPS software used, the processes following the CDR duplicate level checking can differ. Some high-end TPS combines information from various elements to create master CDRs before being loaded into the 'datastore', whilst a lower-end or entry-level TPS directly loads data upon completion of CDR duplicate checking. The 'datastore' used by the TPS consists of a Relational database management system. Some high-end enterprise RDMS includes the likes of ORACLE, Microsoft SQL Server and MySQL. The choice of RDMS to use, is usually driven by company policy, price and recommendations from the TPS supplier. The RDMS should at least be able to support the volumes of CDRs generated by the particular network.

Once all intermediate processing on the CDR batches have completed, the TPS can now load the data into the particular 'datastore'. Separate entities within the RDMS is used to store the data from the different network platforms. The architecture and layout of the RDMS is usually dictated by the particular TPS. The different entities within the RDMS now contains detailed records of each transaction which occurred on any particular platform within the network. Advanced application administrators can now query the detailed data by means of SQL. Depending on network size and subscriber base and particular network platform, the different RDMS entities can become extremely large and queries on these entities requires a high-end hardware architecture. In order to speed up recurring queries and reports on the CDR detail within the RDMS entities, the TPS often summarizes the data based on certain dimensions, also known as aggregates.

Aggregation of loaded data

Summaries, also known as aggregates, are used to summarize important data that needs to be accessed quickly and easily. Recurring reports i.e., hourly, daily and monthly reports, are sourced from aggregates, as retrieving the information from the detail entities can take huge amounts of time and requires a lot of hardware resources to accomplish. The summaries regularly retrieve data from the detail entities within the RDBMS and summarizes it based on certain required information (dimensions). Key measures are summed in order to give hourly, weekly or daily summaries depending on the reporting requirements. Some governments require that at least 6 months of detailed data must be readily available in the RDMS. They also require that a minimum of 5 years of summaries should be available. Due to the high costs of storage, telecommunication organization regularly archive data not required anymore. Data older than the set retention period is moved from high cost storage to low cost permanent media (i.e. tape). Should it be required that data older than the retention period be retrieved, the data can be either restored from the raw CDRs (backed up by the TPS upon collection), or it can be restored from the datastore backups.

Reporting

Management requires reports in order to assess the performance of the business using various KPI's. The data for these reports are processed and stored by the TPS. Summaries are built in order to achieve optimal query and reporting performance. The last function is to present the data in a user friendly, timely and accurate manner. Various tools exists for administrators to present the data to management. BI Tools like ORACLE BI, Business Objects and Cognos can be configured to source data from the TPS datastore summaries giving users the ability to view the data via a web browser. This gives the user up to date information and the ability to cross tabulate and build higher level summaries. Microsoft Excel can also be used to present the data to the end-user. Some TPS providers have the ability of integrating with Microsoft Excel, giving information to the user in a spreadsheet, from where Excel Pivoting can be used to summarize and present the data. Most BI-Tools have functionality to schedule reports which can send reports via e-mail or even SMS to configured users, ensuring timely availability of reports.