Basic Monitoring Recommendations

• Memory utilization
• CPU load
• Disk space
• Disk I/O activity
• Network traffic
• Clock accuracy

• Memory utilization: If your memory utilization is consistently near 75%, consider increasing the memory.
• CPU load: On healthy web application nodes, we typically see CPU load between 0 and 10 (with 10 being high). In your environment, if the CPU load is consistently above 5, you may want to get some thread dumps using the appsnap command, and then open a support case on the Jive Community.
• Disk space: On the web application nodes, you'll need enough disk space for search indexes (which can grow large over time) and for attachment/image/binary content caching. The default limit for the binstore cache is 512MB (configurable from Admin console: System > Settings > Storage Provider). We recommend starting with 512MB for the binstore cache. Note that you also need space for generated static resources.
• Network traffic: While you may not need a specific alert for this, monitoring this is helpful for collecting datapoints. This monitor can be helpful for understanding when traffic dropped off.
• Clock accuracy: In clustered deployments, ensuring the clocks are accurate between web application nodes is critical. We strongly recommend using NTP to keep all of the server clocks in sync.

• Individual web application server
• Through the load balancer's virtual IP address

WebInject interacts with the web application to verify basic functionality. It provides functional tests beyond just connecting to a listening port. Checking individual servers, as well as the load balancer instance, verifies proper load balancer behavior. We recommend setting these checks every five minutes initially. To minimize false alarms, we require two failures before an alert is sent. If you find that these settings are resulting in too many false alarms, then adjust your settings as needed.

For an example of WebInject XML code that will perform all of the above, see WebInject Code Example.

• Java Management Extensions (JMX) hooks (heap)
• Disk space (logs)

• Heap: If your heap is consistently near 75%, consider increasing the heap size. To learn how, be sure to read Adjusting the Java Virtual Machine (JVM) Settings on a Cache Server.

• Connections
• Transactions
• Longest query time and slow queries

Verify ETLs are running

Disk space

Disk I/O activity

• Connections: More connections require more memory. If you're constantly seeing the number of connections spike, consider adding more memory to the database server and make sure that the database server has enough memory to handle the database connections. The number of connections will be a function of what the min/max settings are on each of the web application nodes. (To learn how to set those, see Getting Basic System Information). Out-of-the-box settings for database connections are 25 minimum, 50 maximum. For high-traffic sites in our hosted environment, we set that to 25/125. Note that additional nodes should be used instead of more database connections for managing additional traffic.
• Transactions: If the database provides an easy way to measure this number, it can be helpful for understanding overall traffic volume. However, this metric is less important than monitoring the CPU/memory/IO utilization for capacity planning and alerting.
• Longest query time and slow queries: It's helpful to monitor slow query logs for the database server that they're provisioned against. In our hosted (PostgreSQL) deployments, we log all slow queries (queries that take more than 1000ms seconds) to a file and then monitor those to help find any queries that might be causing issues that could be helped by database indexes.
• Verify ETLs are running: This is important only for the Analytics database. The easiest way to monitor this is by querying the jivedw_etl_job table with something like this: select state, start_ts, end_ts from jivedw_etl_job where etl_job_id = (select max(etl_job_id) from jivedw_etl_job); If the state is 1, the ETL is running. If any state is 3, there is a hard failure that you need to investigate. If the difference between start_ts and end_ts is too big, you may need to increase the resources for the Analytics database.
• Disk space: On the web application nodes, you'll need enough disk space for search indexes (which can grow large over time) and for attachment/image/binary content caching. The default limit for the binstore cache is 512MB (configurable from Admin console: System > Settings > Storage Provider). We recommend starting with 512MB for the binstore cache. Note that you also need space for generated static resources. The most critical place to monitor disk space is on the database server; you should never have less than 50% of your disk available. We recommend setting an alert if you reach more than 50% disk utilization on the database server.
• Disk I/O activity: This is good to record because it can be important if you see slow performance on the web application node(s) and excessive wait time.

• Tomcat I/O
• Heap
• Queue statistics (e.g., average length and wait times)
• Running OpenOffice service statistics
• Overall conversion success rate for each conversion step

• Activity Engine service
• Java Management Extensions (JMX) hooks (heap) and ports
• Queue statistics (e.g., average length and wait times)

• Heap: If your heap is consistently near 75%, consider increasing the heap size. To learn how, be sure to read Adjusting the Java Virtual Machine (JVM) Settings.
• To understand more about the queue depths for the Activity Engine, see Configuring the Activity Engine.