Table of Contents

Overview

Problem Detection/Early Warning System Notification

Performance Reporting

Problem Diagnosis

Installation and Configuration

Working with Third Party connections

Overview
This document explains how use AppMetrics to monitor the health of custom applications built using COM+ and .net Serviced Components on production machines and for troubleshooting problems within the application. AppMetrics is designed to monitor the health of applications in production, enabling system administrators to deliver a higher level of service over more machines, resolve 'finger-pointing' issues, and anticipate system capacity over time.

COM+ components behave differently when under real application load, compared to when they are on the test, QA and especially the developer's local machine. Load factors stress the Application servers, the network, and the database. The monitoring of this real world usage exposes programming, performance, interaction, tuning and scalability issues that the testing world cannot anticipate.

AppMetrics For Transactions effectively monitors the health of COM+ applications without placing a disproportionate amount of load on the system and without requiring changes to your component's source code. In addition, it can be used for detecting bottlenecks and isolating problems.

Problem Detection/Early Warning System/Notification

Introduction
This section explains how to monitor COM+ at run-time and set up an early warning system using the notification features in AppMetrics.

Will AppMetrics for Transactions monitor non-transactional COM+ applications?
The answer is yes! AppMetrics will report on any registered COM+ component, not just those that use COM+ transactions.

While most of the metrics are available for non-transactional components, some transaction-specific metrics, "aborts", for example, work only when the components are marked "transactional".

When we say "Transaction" in this document, we usually mean root object or method.

See Chapter 1 in the AppMetrics for Transactions documentation. In particular, refer to the section The Transactions Measured by AppMetrics. This covers the distinction between components, methods, and transactions.

How to see hung components, methods, or transactions
Suppose a component, method, or transaction, under some circumstances, never returns. Would you even know this is happening? How would you determine what was hung if this occurred on a production machine? AppMetrics for Transactions helps you isolate these conditions right away.

Locate the "In Call" tab in the AppMetrics MMC snap-in.

The "In Call" tab in a Production Manager monitor displays components and transactions listed by name with the time that they were originally called.

In a Diagnostics Manager monitor the "In Call" tab displays components and methods listed by name with the time that they were originally called.

Simply click "Refresh" to update the display with a list of pending components, methods or transactions.

Note any component, method, or transaction that continues to stay around, as these usually indicate a problem.

AppMetrics also alerts you when components don't complete within preset limits. For example, you could configure the product to alert you when components are active for longer than five seconds. A component that exceeds this threshold is either taking a long time to complete or is hanging.

How to get notified when Transactions take too long
It's all about managing expectations. First, decide how long the client to your COM+ component can wait for a given method to return. When it approaches or exceeds this threshold, you want to know.

Use the Notification features to trigger e-mail, SNMP, or Event Log notification whenever the average duration of a component exceeds the specified Benchmark value.

Example: Setting thresholds and getting notified through e-mail
Suppose you are running a mission-critical transaction and want e-mail notification whenever the transaction takes longer than one tenth of a second to complete.

Set up a list of e-mail recipients.

Set thresholds on the transaction

The following graphic shows notification configured on a named transaction called Perform. The duration is set for 100 milliseconds (one-tenth of a second). At 75% the runtime UI will turn yellow, indicating warning. At 100% it will turn red and an error notification will be triggered.

While the Production Manager monitor runs, AppMetrics for Transactions tracks the length of time for each execution of the transaction to commit.

Since the duration exceeds your specified threshold, e-mail notification is sent to the recipients specified.

It's worth noting that in the latter graphic, \\WAUGH is the name of the Agent machine where COM+ is running.

How to determine Transaction thresholds
How do you determine a reasonable threshold?

• Run a Production Manager monitor and Agent monitor while generating load on a healthy system.



• Stop the monitor, go to Benchmarks and Thresholds and examine the Transactions tab. The "Last Recording" box displays the duration taken to complete each Transaction called.



• You'll want to pad these numbers - while a representative load (where the CPU approached 100%) is adequate to determine a threshold, generating such load on an application server would require that you write scripts that loop against your components. Generally, when you load-test a multi-tier application, you'll load test against your web tier (which runs ASP that calls the application server), and this number will reach 100% CPU before the application server does. The correct threshold varies from installation to installation. Available percent CPU on the application server during load test is a good starting point.

Notification for aborted transactions
The Benchmarks and Notification features in AppMetrics for Transactions can trigger e-mail, SNMP, or Event Log notification whenever a Transaction aborts.

As the named transaction called Perform in the following graphic is configured, notification will occur whenever it aborts

Notification when thread counts or memory usage gets high
The Benchmarks and Notification features in AppMetrics for Transactions can trigger e-mail, SNMP, or Event Log notification whenever the number of threads on a COM+ application exceeds a threshold.

To setup e-mail notification

• In a Production Manager monitor, configure your Notifications settings to direct message to the correct SMTP server.
• Specify the recipients of the e-mail messages.
• Set the Threads threshold on the Benchmarks and Thresholds Application/Packages tab.

For example, when thread count exceeds thirty, notification is triggered.

Reporting problem situations into an enterprise management framework
Most management frameworks support Simple Network Management Protocol (SNMP). AppMetrics for Transactions allows you to specify the SNMP community that your management console uses.

Consult your management consoles documentation for more information about SNMP. The key point to remember regarding AppMetrics for Transactions and SNMP is that you can configure AppMetrics for Transactions such that whenever a threshold is crossed, an SNMP trap is triggered.

Correlating processes to Packages/Applications
When you set up and start a monitor with the AppMetrics for Transactions MMC snap-in, note that COM+ Packages/Applications display by name rather than by Process ID (PID). AppMetrics for Transactions lists these by name to save you time and effort.

The COM+ Explorer displays the PID for each COM+ Application. One benefit of running AppMetrics for Transactions is that you can eliminate the manual task of looking up PIDs within the Task Manager to obtain resource use information by process.

Performance Reporting

Introduction
AppMetrics for Transactions can help you demonstrate that Service Level Agreements are being met. It can also collect data that is useful input for capacity planning.

Demonstrating that levels of service are met
First, you must define levels of service. While the specific levels are organization-specific, they are generally based on data such as transaction duration, number of aborts, etc.

The AppMetrics for Transactions Reports produce charts that represent real-time metrics collected over time. These metrics are interactively displayed within Microsoft Excel.

To view AppMetrics for Transactions Reports:

• Simply select Xtremesoft - AppMetrics Reports from the Start menu.
• Specify the dataset desired.
• The dataset is retrieved and reports are automatically generated

Determining the components that use the most computer resources
Consider the transactions taking place within your system: the most expensive are not necessarily the longest. For example, transactions that take a full second but are rarely called are often less of a concern than those taking a half second, but called frequently.

AppMetrics for Transactions Reports helps to address this issue. The "Top 10 Components" report sorts and lists transactions by adding the number of completed and aborted transactions and multiplying that total by the average duration time for each transaction. Using this formula, the "Top 10 Transactions" provides a listing of those transactions most frequently called and those taking the longest to complete.

Problem Diagnosis

Introduction
This section explains how to monitor COM+ at run-time and troubleshoot common problem scenarios using AppMetrics for Transactions. Typically, one takes advantage of the diagnostics features after a notification has occurred in response to a threshold being crossed.

For example, you set up a Production Monitor to notify you via e-mail whenever a particular transaction aborts. Then, you receive such e-mail and configure a Diagnostics Monitor to watch the Package/Application and Component.

What are Diagnostics Monitors?
Diagnostics Monitors track events from the COM+ subsystem and write them to disk. As a result, using Diagnostics Monitors minimize run-time user interface, unlike the interval-based Production Monitors, which keep counts and write to disk at a specified interval (60 seconds by default). Diagnostics Monitors record COM+ events as they are received, immediately writing them to disk.

For example, you use the Diagnostics Manager monitor to locate the bottlenecks in a particular Component. By using the monitor to record each method call as it flows through the system, you can later use AppMetrics for Transaction Reports to examine the results.

Such detail comes at a price. Additional load is placed on the Manager machine (memory, CPU, disk). While the load is usually within acceptable levels it is important to limit the number of Applications/Packages and components monitored, ideally, to a few at a time.

Monitoring component performance over time
To isolate a bottleneck, you want to view performance over time. You can run AppMetrics for Transactions, have it write to a log file, then upload it to a SQL Server database, one for each monitor.

AppMetrics for Transactions includes an Excel file that is configured to report statistics from both Production Monitor and Diagnostics Monitor databases.

Example:
Suppose you configured a Production Monitor so that you receive e-mail notification whenever a particular transaction aborts one or more times during an interval.

• Examine the Excel reports to learn how long the transaction takes, when and whether it does succeed, how often it aborts, when it started aborting, etc.



• Configure a Diagnostics Monitor to monitor the Transaction's Package and Component.



• With the next notification, you can examine the data generated by the Diagnostics Monitor for a more detailed report about the Transaction's performance.

Installation and Configuration

Introduction
This section discusses some of the features that make AppMetrics for Transactions a useful application management tool for large-scale deployments. This section also provides suggestions for best practices in such environments.

Best practices for configuring security in AppMetrics for Transactions
Xtremesoft recommends that you create a W2K/W3K Domain Group called AppMetrics Admins before you deploy AppMetrics for Transactions. By using a Domain Group, you limit access to a single location. During each install, this group and only this group is added to the machine's AppMetrics Administrator's local group and can install the AppMetrics Console or gain access to Excel file reports.

Without a Domain Group, access must be manually granted by adding each AppMetrics console user to the AppMetrics Administrator's local group on each AppMetrics for Transactions Manager and Agent machine. Such manual listings are difficult to set up and maintain.

Providing access to the console and reports
Before you deploy AppMetrics for Transactions, create a W2K/W3K Domain Group called AppMetrics Admins. These users can then choose Reports and Console during the AppMetrics for Transactions install and access the Manager and Agent machines.

Scaling AppMetrics for Transactions Manager installation machines
Memory and CPU affect the number of Manager monitors on a given machine:

Memory Utilization: A minimum of 10 MB RAM should be available for each monitor at runtime.

CPU Utilization:If CPU utilization is high, either reduce the number of Applications/Packages and Components monitored, or reduce the number of Managers running at a given time.

Limiting AppMetrics for Transactions database size
As you establish each monitor's logging setup, you can specify the duration for storing data in the database. The default setting is seven days. Before you create a monitor, determine is how long you want to keep logged data. Modify this setting, if necessary.

Best Practices: Uninstalling AppMetrics for Transactions
During Uninstall, AppMetrics for Transactions attempts to delete each monitor's database. Therefore, if you are using the database elsewhere, back it up before deleting the monitor. Once the monitor is deleted, you can then restore the database from your backup.

Working with AppMetrics for Transactions and NetIQ

Introduction
While many customers use AppMetrics for Transactions as a stand-alone product, it can be tightly integrated with the real-time monitoring and historical analysis features of NetIQ AppManager. The AppMetrics for Transactions setup program provides the following:

• An AppManager extension library facilitates the transfer of AppMetrics for Transactions data into AppManager.
• Knowledge Scripts let you:
• Silently install AppMetrics for Transactions on an AppManager-managed client.
• Generate reports that enable easy access to the data provided by your AppMetrics for Transactions monitors.
• Set event thresholds on the AppMetrics for Transactions data arriving in AppManager. For example, you could set a threshold such that when a particular transaction aborts, or takes a long time, an AppManager event is raised.

The following graphic shows an event being triggered in AppManager when data from AppMetrics for Transactions crosses a threshold set through a knowledge script. The manager machine is named "MARQUEZ", however, the actual message for any event contains the agent machine name (where COM+ is really running).

Installing AppMetrics for Transactions on an AppManager machine
You can install an AppMetrics for Transactions manager on the same machine where you are running AppManager, but you should be sensitive to hardware constraints. Consult AppManager documentation to see how many clients your hardware configuration can manage.

Installing AppMetrics for Transactions will put additional overhead on the AppManager machines and force you to scale out your AppManager machines sooner.