I have a script to collect performance data from vCenter 6.0, here is the script code snippet with comments:
First of all, script gets the metric list for which I want to collect perf data, then it creates queryspecs for all entities (host and VMs) using the following code.
Queryspec for host:
entities = []
host_mor = ManagedObjectReference(value=host_moid, _type="HostSystem")
queryspec = Connection.vim25client.new('PerfQuerySpec', entity=host_mor, metricId=metrics, format=format_type, intervalId=20, startTime=st, endTime=et)
entities.append(queryspec)
Same as queryspec for VMs which are powered on and exist on this host system and append to entities.
Then this queryspec is passed to the performance query method to retrieve performance data.
perfdata = Connection.perfmanager.queryPerf(entities)
The script is working properly in the normal scenario and it is running continuously at every 180 seconds with the 180-seconds difference b/w startTime and endTime. But in some case, it throws the following error:
Serverraisedfault: 'Theobjecthasalreadybeendeletedorhasnotbeencompletelycreated'
If I checked the vpxd logs on vCenter then able to get the log for the same failure at same time with queryspec in log file.
2018-12-05T20:38:28.270+05:30 info vpxd[13140] [Originator@6876 sub=Default opID=63da439b] [VpxLRO] -- ERROR task-internal-127786872 -- PerfMgr -- vim.PerformanceManager.queryStats: vmodl.fault.ManagedObjectNotFound:
--> Result:
--> (vmodl.fault.ManagedObjectNotFound) {
--> faultCause = (vmodl.MethodFault) null,
--> obj = 'vim.VirtualMachine:vm-105',
--> msg = "Received SOAP response fault from [<cs p:0000000025929a20, TCP:<<host_ip>>:443>]: queryBatchPerformanceStatistics
--> "
--> }
--> Args:
-->
--> Arg querySpec:
--> (vim.PerformanceManager.QuerySpec) [
--> (vim.PerformanceManager.QuerySpec) {
--> entity = 'vim.HostSystem:host-513',
--> startTime = "2018-12-05T15:05:21.072535Z",
--> endTime = "2018-12-05T15:08:21.072535Z",
--> maxSample = <unset>,
--> metricId = (vim.PerformanceManager.MetricId) [
--> (vim.PerformanceManager.MetricId) {
--> counterId = 512,
--> instance = ""
--> },
--> (vim.PerformanceManager.MetricId) {
--> counterId = 513,
--> instance = ""
...
Here it gives vm-105 object not found and due to which "Received SOAP response fault" but If I check for the "vm-105" object then it does not exist. For all the failures it gives different objects which do not exist on that server.
Note: I have also logged querspec created by script and it doesn't contain the missing object still vCenter is throwing the error for it.
Can anyone please help when this error occurs and where to get information about this missing object?
