In this post I'll present an example about how to perform custom management measurements in Coherence and then build custom reports based on that measurements.
Example Scenario: In this example I am performing some sort of coherence benchmark. Several operations (insert, update and continuous query) are performed against the coherence cache. I am measuring the time results for this operations and then I am providing the custom Mbeans with the results. Example topology: 1 coherence storage enabled nodes, 1 proxy node, 3 coherence extend clients and 1 management node.Based on the custom MBeans I am building custom reports with Coherence JMX Reporter.
The necessary steps for achieving this are:
- Custom Configuration of Coherence Mbeans
- Crate and configure Coherence Management Node
- Create a Custom Mbean
- create a new package in your project called "mbeans"
- create a generic mbean interface class
- implement the PerformanceMBean interface
- inherit the Performance class for each operation that you want to measure
- add Mbeans classes to the Coherence Management node classpath
- create a custom Mbean XML configuration file
- specify the name and location of a custom MBean configuration file to the coherence management nod
- Connect to the JMX custom Mbeans to update them with the latest results
- Use JConsole to watch your custom Mbean
- Create a custom report
- Localize and extract your custom reports configuration file
- Configure the management node to load your custom report configuration file
- Create and configure a new configuration file for each report that you want to be generated
- Add your new report configuration file to the general Coherence JMX Reporter configuration file
1. Custom Configuration of Coherence Mbeans
1. Create and configure Coherence Management Node
Coherence can be configured to load platform and standard MBeans.To manage a Coherence cluster using JMX, ensure that you have the necessary JMX 1.0 or later classes (javax.management.*) in the classpath of at least one Coherence cluster node, known as an MBeanServer host. The cluster nodes that are not MBeanServer hosts will be managed by the MBeanServer host(s) by using the Coherence Invocation service.
Management node start file:
1: JAVA_OPTS="-Xms$MEMORY -Xmx$MEMORY
2: -Dtangosol.coherence.management.report.configuration=$TEST_HOME/reportsIntel/report-group.xml
3: -Dtangosol.coherence.override=$TEST_HOME/config/tangosol-management-coherence-override-dev.xml
4: -Dtangosol.coherence.management.remote=true;
5: -Dtangosol.coherence.management.report.autostart=true
6: -Dtangosol.coherence.mbeans=mBeans.xml
7: -Dtangosol.coherence.management=all
8: "
9: $JAVAEXEC -server -showversion $JAVA_OPTS -cp
10: "$COHERENCE_HOME/lib/coherence.jar:$TEST_HOME/build/classes:$TEST_HOME/config:$TEST_HOME/lib/jmxri.jar:$TEST_HOME/lib/jmxtools.jar"
11: com.tangosol.net.management.MBeanConnector -rmi
com.tangosol.net.management.MBeanConnector -rmi = Start a Coherence cluster node that enables JMX agents such as a web browser or JConsole to connect to the in-process JMX MBeanServer and manage Coherence MBeans. -rmi is used when connecting via Jconsole.
2. Create a Custom Mbean
For more information about standard Mbeans please access the following documentation:
http://download.oracle.com/javase/tutorial/jmx/mbeans/standard.html
In the following example you will see how I've created a custom MBean for measuring insert, update and continuous query operations:
1. create a new package in your project: "mbeans" for instance
2. create a generic mbean interface class called PerformanceMBean that describes what are the measurables for a certain operation:
1: /**
2: *
3: */
4: package com.oracle.coherence.test.mbeans;
5:
6: /**
7: * @author tucosmin
8: *
9: */
10: public interface PerformanceMBean
11: {
12: public int getTotalNoOfThisOperation();
13:
14: public void setTotalNoOfThisOperation(int totalNoOfThisOperation);
15:
16: public int getIntermTotalNoOfThisOperation();
17:
18: public void setIntermTotalNoOfThisOperation(int intermTotalNoOfThisOperation);
19:
20: public int getIntermTotalNoOfOperations();
21:
22: public void setIntermTotalNoOfOperations(int intermTotalNoOfOperations);
23:
24: public double getIntermAverageOperationDuration();
25:
26: public void setIntermAverageOperationDuration(
27: double intermAverageOperationDuration);
28:
29: public double getIntermMinOperationDuration();
30:
31: public void setIntermMinOperationDuration(double intermMinOperationDuration);
32:
33: public double getIntermMaxOperationDuration();
34:
35: public void setIntermMaxOperationDuration(double intermMaxOperationDuration);
36:
37: public int getTotalNoOfOperations();
38:
39: public void setTotalNoOfOperations(int totalNoOfOperations);
40:
41: public double getAverageOperationDuration();
42:
43: public void setAverageOperationDuration(double averageOperationDuration);
44:
45: public double getMinOperationDuration();
46:
47: public void setMinOperationDuration(double minOperationDuration);
48:
49: public double getMaxOperationDuration();
50:
51: public void setMaxOperationDuration(double maxOperationDuration);
52:
53: public String getClient();
54:
55: public void setClient(String client);
56:
57: public String getNumberOfOperationsPerTimeUnit();
58:
59: public void setNumberOfOperationsPerTimeUnit(
60: String numberOfOperationsPerTimeUnit);
61: }
3. implement the PerformanceMBean interface:
1: /**
2: *
3: */
4: package com.oracle.coherence.test.mbeans;
5:
6: /**
7: * @author tucosmin
8: *
9: */
10: public class Performance implements PerformanceMBean
11: {
12: public int totalNoOfThisOperation;
13: public int totalNoOfOperations;
14: public double averageOperationDuration;
15: public double minOperationDuration;
16: public double maxOperationDuration;
17: public int intermTotalNoOfThisOperation;
18: public int intermTotalNoOfOperations;
19: public double intermAverageOperationDuration;
20: public double intermMinOperationDuration;
21: public double intermMaxOperationDuration;
22: public String numberOfOperationsPerTimeUnit;
23: public String client;
24:
25: public int getTotalNoOfThisOperation()
26: {
27: return totalNoOfThisOperation;
28: }
29:
30: public void setTotalNoOfThisOperation(int totalNoOfThisOperation)
31: {
32: this.totalNoOfThisOperation = totalNoOfThisOperation;
33: }
34:
35: public int getIntermTotalNoOfThisOperation()
36: {
37: return intermTotalNoOfThisOperation;
38: }
39:
40: public void setIntermTotalNoOfThisOperation(int intermTotalNoOfThisOperation)
41: {
42: this.intermTotalNoOfThisOperation = intermTotalNoOfThisOperation;
43: }
44:
45: public int getIntermTotalNoOfOperations()
46: {
47: return intermTotalNoOfOperations;
48: }
49:
50: public void setIntermTotalNoOfOperations(int intermTotalNoOfOperations)
51: {
52: this.intermTotalNoOfOperations = intermTotalNoOfOperations;
53: }
54:
55: public double getIntermAverageOperationDuration()
56: {
57: return intermAverageOperationDuration;
58: }
59:
60: public void setIntermAverageOperationDuration(
61: double intermAverageOperationDuration)
62: {
63: this.intermAverageOperationDuration = intermAverageOperationDuration;
64: }
65:
66: public double getIntermMinOperationDuration()
67: {
68: return intermMinOperationDuration;
69: }
70:
71: public void setIntermMinOperationDuration(double intermMinOperationDuration)
72: {
73: this.intermMinOperationDuration = intermMinOperationDuration;
74: }
75:
76: public double getIntermMaxOperationDuration()
77: {
78: return intermMaxOperationDuration;
79: }
80:
81: public void setIntermMaxOperationDuration(double intermMaxOperationDuration)
82: {
83: this.intermMaxOperationDuration = intermMaxOperationDuration;
84: }
85:
86: public int getTotalNoOfOperations()
87: {
88: return totalNoOfOperations;
89: }
90:
91: public void setTotalNoOfOperations(int totalNoOfOperations)
92: {
93: this.totalNoOfOperations = totalNoOfOperations;
94: }
95:
96: public double getAverageOperationDuration()
97: {
98: return averageOperationDuration;
99: }
100:
101: public void setAverageOperationDuration(double averageOperationDuration)
102: {
103: this.averageOperationDuration = averageOperationDuration;
104: }
105:
106: public double getMinOperationDuration()
107: {
108: return minOperationDuration;
109: }
110:
111: public void setMinOperationDuration(double minOperationDuration)
112: {
113: this.minOperationDuration = minOperationDuration;
114: }
115:
116: public double getMaxOperationDuration()
117: {
118: return maxOperationDuration;
119: }
120:
121: public void setMaxOperationDuration(double maxOperationDuration)
122: {
123: this.maxOperationDuration = maxOperationDuration;
124: }
125:
126: public String getClient()
127: {
128: return client;
129: }
130:
131: public void setClient(String client)
132: {
133: this.client = client;
134: }
135:
136: public String getNumberOfOperationsPerTimeUnit()
137: {
138: return numberOfOperationsPerTimeUnit;
139: }
140:
141: public void setNumberOfOperationsPerTimeUnit(
142: String numberOfOperationsPerTimeUnit)
143: {
144: this.numberOfOperationsPerTimeUnit = numberOfOperationsPerTimeUnit;
145: }
146: }
4. inherit the Performance class for each operation that you want to measure:
1: /**
2: *
3: */
4: package com.oracle.coherence.test.mbeans;
5:
6: /**
7: * @author tucosmin
8: *
9: */
10: public class UpdatePerformance extends Performance
11: {
12: public UpdatePerformance()
13: {
14: super();
15: // TODO Auto-generated constructor stub
16: }
17: }
1: /**
2: *
3: */
4: package com.oracle.coherence.test.mbeans;
5:
6: /**
7: * @author tucosmin
8: *
9: */
10: public class WritePerformance extends Performance
11: {
12: public WritePerformance()
13: {
14: super();
15: // TODO Auto-generated constructor stub
16: }
17: }
5. add Mbeans classes to the Coherence Management node classpath:
-cp $TEST_HOME/build/classes
6. create a custom Mbean XML file
1: <mbeans>
2: <mbean id="100">
3: <mbean-class>com.oracle.coherence.test.mbeans.WritePerformance
4: </mbean-class>
5: <mbean-name>type=Performance,name=WritePerformance</mbean-name>
6: <enabled>true</enabled>
7: </mbean>
8: <mbean id="101">
9: <mbean-class>com.oracle.coherence.test.mbeans.ContinuousQueryInsertPerformance
10: </mbean-class>
11: <mbean-name>type=Performance,name=ContinuousQueryInsertPerformance
12: </mbean-name>
13: <enabled>true</enabled>
14: </mbean>
15: <mbean id="102">
16: <mbean-class>com.oracle.coherence.test.mbeans.ContinuousQueryUpdatePerformance
17: </mbean-class>
18: <mbean-name>type=Performance,name=ContinuousQueryUpdatePerformance
19: </mbean-name>
20: <enabled>true</enabled>
21: </mbean>
22: <mbean id="103">
23: <mbean-class>com.oracle.coherence.test.mbeans.UpdatePerformance
24: </mbean-class>
25: <mbean-name>type=Performance,name=UpdatePerformance
26: </mbean-name>
27: <enabled>true</enabled>
28: </mbean>
29: </mbeans>
30:
7. specify the name and location of a custom MBean configuration file to the coherence management node:
-Dtangosol.coherence.mbeans=mBeans.xml
3. Connect to the JMX custom Mbeans to update them with the latest results
---
com.oracle.coherence.jmxUrl=service:jmx:rmi://tucosmin-ro:3000/jndi/rmi://tucosmin-ro:9000/server
---
1: public void sendOperationPerformanceToJmx(String jmxPath,
2: ReportOutput reportOutput)
3: {
4: if (reportOutput != null)
5: {
6: // Create a JMXServiceURL
7: JMXServiceURL serviceUrl;
8: try
9: {
10: serviceUrl = new JMXServiceURL(DEFAULT_JMX_SERVER_URL);
11: JMXConnector connector = JMXConnectorFactory.newJMXConnector(
12: serviceUrl, null);
13: connector.connect();
14: // Create an MBeanServerConnection
15: MBeanServerConnection connection = connector
16: .getMBeanServerConnection();
17: ObjectName name = new ObjectName(jmxPath);
18: Set<ObjectName> nodes = connection.queryNames(name, null);
19: for (Object o : nodes)
20: {
21: Double average = 0.0;
22: int count = 0;
23: if (reportOutput.getOpperation().equalsIgnoreCase(
24: Constants.WRITE_TIME))
25: {
26: average = getWriteAverage().getAverage();
27: count = getWriteAverage().getCount();
28: }
29: else
30: if (reportOutput.getOpperation().equalsIgnoreCase(
31: Constants.CONTINUOUS_QUERY_INSERT_TIME))
32: {
33: average = getContinuousQueryInsertAverage()
34: .getAverage();
35: count = getContinuousQueryInsertAverage()
36: .getCount();
37: }
38: else
39: if (reportOutput.getOpperation().equalsIgnoreCase(
40: Constants.CONTINUOUS_QUERY_UPDATE_TIME))
41: {
42: average = getContinuousQueryUpdateAverage()
43: .getAverage();
44: count = getContinuousQueryUpdateAverage()
45: .getCount();
46: }
47: else
48: if (reportOutput
49: .getOpperation()
50: .equalsIgnoreCase(Constants.UPDATE_TIME))
51: {
52: average = getUpdateAverage().getAverage();
53: count = getUpdateAverage().getCount();
54: }
55: connection.setAttribute(new ObjectName(o.toString()),
56: new Attribute(Constants.AVERAGE_OPERATION_DURATION,
57: average));
58: int totalNoOfThisOp = Integer.parseInt(connection
59: .getAttribute(new ObjectName(o.toString()),
60: Constants.TOTAL_NO_OF_THIS_OPERATION)
61: .toString());
62: totalNoOfThisOp += reportOutput.getTotalNoOfOperations();
63: connection.setAttribute(new ObjectName(o.toString()),
64: new Attribute(Constants.TOTAL_NO_OF_THIS_OPERATION,
65: count));
66: connection.setAttribute(new ObjectName(o.toString()),
67: new Attribute(Constants.TOTAL_NO_OF_OPERATIONS,
68: getTotalNoOfOperations()));
69: Double minOperationDuration = new Double(connection
70: .getAttribute(new ObjectName(o.toString()),
71: Constants.MIN_OPERATION_DURATION)
72: .toString());
73: Double maxOperationDuration = new Double(connection
74: .getAttribute(new ObjectName(o.toString()),
75: Constants.MAX_OPERATION_DURATION)
76: .toString());
77: if (minOperationDuration == 0)
78: {
79: connection.setAttribute(new ObjectName(o.toString()),
80: new Attribute(Constants.MIN_OPERATION_DURATION,
81: reportOutput.minOperationDuration));
82: }
83: else
84: {
85: if (minOperationDuration > reportOutput.minOperationDuration
86: && reportOutput.minOperationDuration > 0)
87: {
88: connection.setAttribute(
89: new ObjectName(o.toString()),
90: new Attribute(
91: Constants.MIN_OPERATION_DURATION,
92: reportOutput.minOperationDuration));
93: }
94: else
95: {
96: if (minOperationDuration > 0)
97: {
98: connection
99: .setAttribute(
100: new ObjectName(o.toString()),
101: new Attribute(
102: Constants.MIN_OPERATION_DURATION,
103: minOperationDuration));
104: }
105: }
106: }
107: if (maxOperationDuration < reportOutput.maxOperationDuration)
108: {
109: connection.setAttribute(new ObjectName(o.toString()),
110: new Attribute(Constants.MAX_OPERATION_DURATION,
111: reportOutput.maxOperationDuration));
112: }
113: else
114: {
115: connection.setAttribute(new ObjectName(o.toString()),
116: new Attribute(Constants.MAX_OPERATION_DURATION,
117: maxOperationDuration));
118: }
119: connection.setAttribute(new ObjectName(o.toString()),
120: new Attribute(
121: Constants.INTERM_MAX_OPERATION_DURATION,
122: reportOutput.getMaxOperationDuration()));
123: connection
124: .setAttribute(
125: new ObjectName(o.toString()),
126: new Attribute(
127: Constants.INTERM_TOTAL_NO_OF_THIS_OPERATION,
128: reportOutput
129: .getTotalNoOfOperations()));
130: connection
131: .setAttribute(
132: new ObjectName(o.toString()),
133: new Attribute(
134: Constants.INTERM_AVERAGE_OPERATION_DURATION,
135: reportOutput
136: .getAverageOperationDuration()));
137: if (reportOutput.getMinOperationDuration() > 0)
138: {
139: connection
140: .setAttribute(
141: new ObjectName(o.toString()),
142: new Attribute(
143: Constants.INTERM_MIN_OPERATION_DURATION,
144: reportOutputsendOperationPerformanceToJmx
145: .getMinOperationDuration()));
146: }
147: connection.setAttribute(new ObjectName(o.toString()),
148: new Attribute(
149: Constants.INTERM_TOTAL_NO_OF_OPERATIONS,
150: getResults().size()));
151: connection.setAttribute(new ObjectName(o.toString()),
152: new Attribute(Constants.CLIENT, "Client "
153: + Constants.COHERENCE_EXTEND_CLIENT_ID));
154: }
155: }
156: catch (Exception e)
157: {
158: // TODO Auto-generated catch block
159: e.printStackTrace();
160: }
161: }
162: }
4. Use JConsole to watch your custom Mbeans
II. Create a custom report
The Coherence reporting feature provides a capable query definition that allows for any information residing in the Coherence JMX data source to be logged to a text file. After a custom report has been created, it can be included in a report batch and executed on a specified time interval by the
ReportControl Mbean.Coherence provides a JMX reporting capability (the Reporter). The Reporter provides out-of-the-box reports that help administrators and developers manage capacity and trouble shoot problems. Custom reports can also be created.
1. Localize and extract your custom reports configuration files:
- Go to your COHERENCE_HOME/lib location, go to the archive coherence.jar and extract from there into your project folder the reports folder. In that folder you should find a report-group.xml file together with other configuration files
2. Configure the management node to load your custom report configuration files
- In your management node start file specify from where the management node could access the reports configuration files: -Dtangosol.coherence.management.report.configuration=$TEST_HOME/reportsIntel/report-group.xml Without this specification the reports configuration files are loaded from the default location: coherence.jar\reports. You can modify the configuration from that location as well but the best practice is to copy them in your project location.
3. Create and configure a new configuration file for each report that you want to be generated
- Create an xml file for each report that you want to be generated. As an example you can see the following xml report configuration file for the insert operation:
1: <!DOCTYPE report-config SYSTEM "report-config.dtd">
2: <report-config>
3: <report>
4: <file-name>{date}-total-write-performance.txt</file-name>
5: <delim>{tab}</delim>
6: <query>
7: <pattern>Coherence:type=Performance,name=WritePerformance,*</pattern>
8: </query>
9: <row>
10: <column id="BatchCounter">
11: <type>global</type>
12: <name>{batch-counter}</name>
13: <header>Batch Counter</header>
14: </column>
15: <column id="ReportTime">
16: <type>global</type>
17: <name>{report-time}</name>
18: <header>Report Time</header>
19: </column>
20: <column id="MinOperationDuration">
21: <name>MinOperationDuration</name>
22: <header>Total(until now) Minimum Write Operation Duration</header>
23: </column>
24: <column id="MaxOperationDuration">
25: <name>MaxOperationDuration</name>
26: <header>Total(until now) Maximum Write Operation Duration</header>
27: </column>
28: <column id="AverageOperationDuration">
29: <name>AverageOperationDuration</name>
30: <header>Total(until now) Write Average Operation Duration</header>
31: </column>
32: <column id="TotalNoOfOperations">
33: <name>TotalNoOfOperations</name>
34: <header>Total Number of All Operations Until Now</header>
35: </column>
36: <column id="TotalNoOfThisOperation">
37: <name>TotalNoOfThisOperation</name>
38: <header>Total Number Of Write Operations Until Now</header>
39: </column>
40: <column id="NumberOfOperationsPerTimeUnit">
41: <name>NumberOfOperationsPerTimeUnit</name>
42: <header>Number Of Write Operations per TimeUnit</header>
43: </column>
44: <column id="Client">
45: <name>Client</name>
46: <header>Client</header>
47: </column>
48: </row>
49: </report>
50: </report-config>
The element <file-name> defines the reports file name. As you can see it includes the the keyword date which means that every report name will contain the creation date.
The next element defines the Mbean that will be used in your custom report:
1: <query>
2: <pattern>Coherence:type=Performance,name=WritePerformance,*</pattern>
3: </query>
Then use your Mbean attributes and report default attributes to build your custom report:
- Report default attributes:
1: <column id="ReportTime">
2: <type>global</type>
3: <name>{report-time}</name>
4: <header>Report Time</header>
5: </column>
- Mbean attributes:
1: <column id="MaxOperationDuration">
2: <name>MaxOperationDuration</name>
3: <header>Total(until now) Maximum Write Operation Duration</header>
4: </column>
4. Add your new report configuration file to the general Coherence JMX Reporter configuration file:
- Add your custom report xml configuration file to the report-group.xml Coherence JMX Reporter configuration file:
1: <!DOCTYPE report-group SYSTEM "report-group.dtd">
2: <report-group>
3: <frequency>30s</frequency>
4: <output-directory>/media/Software/reports/</output-directory>
5: <report-list>
6: <report-config>
7: <location>/media/Software/report-running-write-performance.xml</location>
8: </report-config>
9: <report-config>
10: <location>/media/Software/report-running-update-performance.xml</location>
11: </report-config>
12: <report-config>
13: <location>/media/Software/report-running-continuous-query-insert-performance.xml</location>
14: </report-config>
15: <report-config>
16: <location>/media/Software/report-running-continuous-query-update-performance.xml</location>
17: </report-config>
18: <report-config>
19: <location>/media/Software/report-total-write-performance.xml</location>
20: </report-config>
21: <report-config>
22: <location>/media/Software/report-total-update-performance.xml</location>
23: </report-config>
24: <report-config>
25: <location>/media/Software/report-total-continuous-query-insert-performance.xml</location>
26: </report-config>
27: <report-config>
28: <location>/media/Software/report-total-continuous-query-update-performance.xml</location>
29: </report-config>
30: </report-list>
31: </report-group>
32:
The following can be also specified:
- reporting frequency: <frequency>30s</frequency>
- The reports output directory: <output-directory>/media/Software/reports/</output-directory>
Report example (the reports are generated as txt documents by they are comma separated, so they are csv actually):
No comments:
Post a Comment