Proactive data mining
Like other industry verticals the telecom sector depends on the execution of complex business processes. Usually business processes include interactions among multiple heterogeneous IT systems and human resources. The efficiency and success of these business processes in an operational environment can be monitored based on multiple Key Performance Indicators (KPIs) of the business such as customer satisfaction, overall business performance and adaptability to changes. As a reason it is becoming important to monitor the performance of the business processes and optimize them based on the results. Presently businesses come to know about a problem when a failure occurs in the business processes. Monitoring is one of the best ways to sense an imminent problem and avoid it with corrective actions in advance so that, the impact on business and the users can be minimized.
This article will try to present a solution to monitor and optimize the business processes based on the currently available technologies and their maturity.
Purpose of Business Activity Monitoring
The term business activity monitoring was introduced by Gartner, Inc. in 2001 as their notion of aggregation, analysis, and real-time presentation functionality of business relevant information. The main objective of business activity monitoring is to predict or sense the upcoming problem based on the operational transactions and report the same for a preventive measure.
The term business activity monitoring was introduced by Gartner, Inc. in 2001 as their notion of aggregation, analysis, and real-time presentation functionality of business relevant information. The main objective of business activity monitoring is to predict or sense the upcoming problem based on the operational transactions and report the same for a preventive measure.
It is becoming increasingly significant in today’s enterprises to consider informed decisions in timely manner by consolidating information present in various silo data sources. These decisions eventually have major impact on the business parameters such as profitability, revenue, customer satisfaction and agility. IT systems like Enterprise Resource Planning (ERP), Customer Relationship Management (CRM), Order Management System (OMS), Business Process Management (BPM), etc. have served with considerable efficiencies in their own areas of the business. However mostly the business processes span across multiple systems.
If we imagine of an enterprise with the capability of foreseeing the next steps, predicting impact of events, sensing an unseen fault by putting two and two together from different systems and eventually spot and plug any hole in the plumbing before the damage is done, a tremendous agility, resilience and efficiency in the business processes can be introduced.
A layer that is able to monitor business processes in the above mentioned manner should be able to understand and perform real time monitoring of vital signs in the context of business process. This monitoring should be made proactive as much as possible. Apart from the failures, the deviations in the normal trends based on the historical data need to be flagged and alerted to appropriate stakeholders with minimum time lag. Among the on-demand features, it should have ad hoc reports, dashboards for the decision making community within the enterprise. Such a layer would bring in manifold benefits.
For example, let’s consider an e-commerce scenario with the key business process of fulfillment of orders placed online. The customer places an order with a set of items and gets notified with a date of delivery. Under a happy path scenario, the customer receives the items on the promised date. Considering a variation when the logistics of delivery such as trucks, delivery boys etc. are operating at the full capacity due to a sudden increase in the number orders placed in, an individual order might get delayed in the delivery. The activity monitoring layer could potentially correlate the high volumes being handled by the system to a potential delay in delivery times. This can trigger an alert or communication to customers informing them about the delay before it actually takes place.
Imagine a more interesting scenario of inventory management for the e-commerce site. Assuming that there is a forecast for thunderstorms for couple of days in some of the areas, the monitoring layer can send an alert to the inventory management group to stock up on rainy weather essentials such as umbrellas and raincoats to enable the business to meet the un-foreseen increased demand.
Components of a Business Activity Monitoring
BAM consists of the following generic components:
Business Event
Key performance indicator
KPIs are the metrics used to measure the progress of a business goal. KPIs have to be measurable as well as achievable and provide the possibility to aggregate measures over a set of instances. In order to define KPIs, companies should have clear performance requirements, and they must also have a clear definition of their strategic target which can be compared to runtime values.
If we imagine of an enterprise with the capability of foreseeing the next steps, predicting impact of events, sensing an unseen fault by putting two and two together from different systems and eventually spot and plug any hole in the plumbing before the damage is done, a tremendous agility, resilience and efficiency in the business processes can be introduced.
A layer that is able to monitor business processes in the above mentioned manner should be able to understand and perform real time monitoring of vital signs in the context of business process. This monitoring should be made proactive as much as possible. Apart from the failures, the deviations in the normal trends based on the historical data need to be flagged and alerted to appropriate stakeholders with minimum time lag. Among the on-demand features, it should have ad hoc reports, dashboards for the decision making community within the enterprise. Such a layer would bring in manifold benefits.
For example, let’s consider an e-commerce scenario with the key business process of fulfillment of orders placed online. The customer places an order with a set of items and gets notified with a date of delivery. Under a happy path scenario, the customer receives the items on the promised date. Considering a variation when the logistics of delivery such as trucks, delivery boys etc. are operating at the full capacity due to a sudden increase in the number orders placed in, an individual order might get delayed in the delivery. The activity monitoring layer could potentially correlate the high volumes being handled by the system to a potential delay in delivery times. This can trigger an alert or communication to customers informing them about the delay before it actually takes place.
Imagine a more interesting scenario of inventory management for the e-commerce site. Assuming that there is a forecast for thunderstorms for couple of days in some of the areas, the monitoring layer can send an alert to the inventory management group to stock up on rainy weather essentials such as umbrellas and raincoats to enable the business to meet the un-foreseen increased demand.
Components of a Business Activity Monitoring
BAM consists of the following generic components:
Business Event
A business event points to the occurrence of an event that is significant to an operation of a business activity. It is separate from an IT event. An IT event usually shows the occurrence of an event in the context of an IT resource.
Business activity
A business activity offers services or functionalities required to serve a specific business purpose.
Metric
A metric is a measurement of a specific property belonging to business activity that can be used to monitor business operations.
Key performance indicator
KPIs are the metrics used to measure the progress of a business goal. KPIs have to be measurable as well as achievable and provide the possibility to aggregate measures over a set of instances. In order to define KPIs, companies should have clear performance requirements, and they must also have a clear definition of their strategic target which can be compared to runtime values.
Techniques of Today
Though BAM is relatively a new term are some techniques that have been around for a while. The stalwarts amongst these are
• IT system monitors
• Business Intelligence (BI)
• Reporting and Analytics
• Business Process Management (BPM)
Each of the above has its own pros and cons when the BAM features are evaluated. Let us dig a little deeper in these three techniques.
IT System Monitors: To be able to monitor failures and success events at times many enterprises have implemented IT system monitoring measures in effective ways. The main objective of such systems is to alert the support services in a timely manner to meet the service level of agreement and availability levels of the application. This technique targets individual applications for monitoring and helps them to react under a error condition at the application level. While it does not consider the entire business process as a monitoring target as a result it is unable to track any fault at the business process level. IT system monitors are mostly reactive than proactive. They keep watching the applications, logs, network, server, hardware in isolation for faults rather than the end-to-end business process. IT system monitors will be unable to detect an error at the business process level while the applications are running smoothly. IT system monitors watch for the malfunctions and failures only, while an ideal BAM should look for success and failures both to yield meaningful activity metrics. They are also not aware of the business process level KPIs (Key Performance Indicators).
For example, if a customer is promised with the certain duration for an order fulfillment process, a KPI needs to be set at the business process level for order processing. IT system monitors are unable to detect a degradation of this kind of KPIs. The scope of alerts is limited to the application and does not correlate to failures in other participating application in the same process. It is also unable to detect any variation in the business process when the applications are functioning properly.
In spite of all shortcomings these tools have been serving as monitoring instruments for many organizations. In fact there have been conscious design efforts introduced to log success events (along with the failures) in individual applications’ logs against a unique correlation id and also a listener mechanism to poll the logs in a timely manner and correlate the events using the unique id for each business process transactions. However since the number of I/O operations increase (due to write in log and timely polling of listeners), the performance of the applications is compromised seriously under this situation.
Business Intelligence (BI): Business Intelligence has been occupying the place of a backbone for decision making process of the organizations for quite some time now. This is due to its capabilities of analytics and extensive reporting. BI is capable of processing enormous amount of data stored in huge data marts at predefined frequencies by using ETL (Extraction Transformation and Loading) tools and put the same into the data stores which gets reported eventually. This way BI enables businesses to make informed decisions to a great extent. However when it comes to BAM some constraints apply.
• Since BI and data warehousing are based on batch processes, proactive and real time decisions can not be made easily using BI.
• The statistical analysis is performed manually in BI. Since the monitoring framework is not updated with the historical trends, it’s not able to flag events which are deviating from the normal trend.
• Since BI is not a real time system, it can not track real time errors or deviations and generate alerts for that.
Business Process Management (BPM): Enterprises use business process management to manage, model and automate business processes. Externalization of all business processes those are entrenched deeply in applications to a central location is one of the main tenets of using BPM. BPM accomplishes quite a lot from that angle in terms of improved process agility and centralized governance of vital processes. But BPM also has some shortcomings as a BAM solution:
• BPM has no built in monitoring capabilities. Therefore, alerts based on the success or failure of a process can not be generated by the BPM infrastructure. Furthermore the success and failures can not be evaluated against the historical trend to compose intelligent reports necessary for the right changes or decisions to the activities.
• BPM has the knowledge of a business process while it is lacking the knowledge of relationship between the participating applications, historical data and statistical analysis which are required for anticipating the unknown errors in a business process.
The table below presents a comparative analysis of the BAM techniques described above. It also shows the rating of the three techniques as an ideal BAM solution.
• IT system monitors
• Business Intelligence (BI)
• Reporting and Analytics
• Business Process Management (BPM)
Each of the above has its own pros and cons when the BAM features are evaluated. Let us dig a little deeper in these three techniques.
IT System Monitors: To be able to monitor failures and success events at times many enterprises have implemented IT system monitoring measures in effective ways. The main objective of such systems is to alert the support services in a timely manner to meet the service level of agreement and availability levels of the application. This technique targets individual applications for monitoring and helps them to react under a error condition at the application level. While it does not consider the entire business process as a monitoring target as a result it is unable to track any fault at the business process level. IT system monitors are mostly reactive than proactive. They keep watching the applications, logs, network, server, hardware in isolation for faults rather than the end-to-end business process. IT system monitors will be unable to detect an error at the business process level while the applications are running smoothly. IT system monitors watch for the malfunctions and failures only, while an ideal BAM should look for success and failures both to yield meaningful activity metrics. They are also not aware of the business process level KPIs (Key Performance Indicators).
For example, if a customer is promised with the certain duration for an order fulfillment process, a KPI needs to be set at the business process level for order processing. IT system monitors are unable to detect a degradation of this kind of KPIs. The scope of alerts is limited to the application and does not correlate to failures in other participating application in the same process. It is also unable to detect any variation in the business process when the applications are functioning properly.
In spite of all shortcomings these tools have been serving as monitoring instruments for many organizations. In fact there have been conscious design efforts introduced to log success events (along with the failures) in individual applications’ logs against a unique correlation id and also a listener mechanism to poll the logs in a timely manner and correlate the events using the unique id for each business process transactions. However since the number of I/O operations increase (due to write in log and timely polling of listeners), the performance of the applications is compromised seriously under this situation.
Business Intelligence (BI): Business Intelligence has been occupying the place of a backbone for decision making process of the organizations for quite some time now. This is due to its capabilities of analytics and extensive reporting. BI is capable of processing enormous amount of data stored in huge data marts at predefined frequencies by using ETL (Extraction Transformation and Loading) tools and put the same into the data stores which gets reported eventually. This way BI enables businesses to make informed decisions to a great extent. However when it comes to BAM some constraints apply.
• Since BI and data warehousing are based on batch processes, proactive and real time decisions can not be made easily using BI.
• The statistical analysis is performed manually in BI. Since the monitoring framework is not updated with the historical trends, it’s not able to flag events which are deviating from the normal trend.
• Since BI is not a real time system, it can not track real time errors or deviations and generate alerts for that.
Business Process Management (BPM): Enterprises use business process management to manage, model and automate business processes. Externalization of all business processes those are entrenched deeply in applications to a central location is one of the main tenets of using BPM. BPM accomplishes quite a lot from that angle in terms of improved process agility and centralized governance of vital processes. But BPM also has some shortcomings as a BAM solution:
• BPM has no built in monitoring capabilities. Therefore, alerts based on the success or failure of a process can not be generated by the BPM infrastructure. Furthermore the success and failures can not be evaluated against the historical trend to compose intelligent reports necessary for the right changes or decisions to the activities.
• BPM has the knowledge of a business process while it is lacking the knowledge of relationship between the participating applications, historical data and statistical analysis which are required for anticipating the unknown errors in a business process.
The table below presents a comparative analysis of the BAM techniques described above. It also shows the rating of the three techniques as an ideal BAM solution.
Table 1: Comparative analysis of three BAM techniques
No comments:
Post a Comment