I don’t see recent data but data from some hours ago.
That´s a very common sign of a wrong time of the Performance analyzer appliance. That also means that the ESXi host time the appliance runs on is wrong.
The Performance Analyzer appliance contains and runs VMware Tools configured for time synchronization with the ESXi host.
Therefore, a wrong time of the appliance is a typical sign of a wrong time on the ESXi host. Please check your ESXi hosts time and NTP settings and adjust it. It makes sense to restart the Performance Analyzer appliance.
There is no new data in the dashboards anymore and most charts are empty
Disk Space is full
If the system partition is close or at 100% please let increase the size of the first virtual disk of the Performance Analyzer appliance. There is no need to shutdown the appliance, as we auto-resize the partition within 5 minutes.
If the data partition is close or at 100% please let increase the size of the second virtual disk of the Performance Analyzer appliance. There is no need to shutdown the appliance, as we auto-resize the partition within 5 minutes.
VMware dashboards suddenly stop to show new data
VMware vCenter user credentials have changed
Please check if the vCenter connection is still working (Admin Web Interface / http://appliance-ip/admin) and a green state including the number of ESXi hosts is shown. You can click the test button to re-test the connection. Use the edit button to set the new password.
If the vCenter has been renamed, you can change that the same way
random gaps in graphs, lasting between 5 minutes and several hours
There are three main causes, which can be responsible for the problem:
The gaps are the result of the appliances automatic mitigation for this situation: Whenever the timeseries databases get unresponsive due to too much outstanding IOPS, they will be restarted, purging the unhandled transactions.
You can improve the situation either by moving the appliance to faster storage, reduce noisy neighbors or increase the appliance memory to have the time series databases in the OS disk cache.
Some words on the latter: Since retention data is calculated before writing metrics to disk, the time series databases need to do a lot of reads, to gather all data for updating the samples in the higher retentions. As long as the OS cache is big enough, these IO won’t hit the storage, so there is only the write IO, which will go to storage.
But when the OS disk cache is even slightly smaller than the time series database working set, the cache will get continuously revolved on each crawler run. So in case you suffer from that continuous revolving, you should increase the appliances memory, until the OS disk cache is big enough.
Some hints for sizing the appliance memory and observing the IOPS:
Open the dashboard “VMware Performance: VM”, select 3 hour interval, and choose your Performance Analyzer VM. Then scroll down and open the row “vDisk Metrics”. If there are more Read IOPS then Write IOPS you should try the following procedure:
Determine the time series database footprint:
Open the dashboard “VMware Performance: VM”, select 3 hour interval and choose your Performance Analyzer VM. Scroll down and open the row “vDisk Metrics”:
In this example you see an appliance which got memory doubled at 15:30 and was powercyled for that. Everything before 15:30 shows the appliance running with to few memory, leading to continuous cache revolving and lots of disk reads. After 15:30 the cache is big enough. It fills within about 15 minutes after reboot, then read IO drops nearly to zero.
When the main memory is chosen big enough, you will see almost no Disk Reads. Also read IOPS will almost vanish. Disk Write and Write IOs will be swinging more, with a slight increase of the mean value. This is because of the much eager write back performance, so the likelihood for a block to be written several times in the cache before eventually getting written to disk sinks. However, overall you should see a massive reduction of the IO footprint, when main memory is big enough.