I am working on monitoring the performance of an Azure service.
There are currently two web role instances (for the same website) running - each with its own W3WP.exe (w3wp and w3wp#1)
How can i find out which w3wp process belongs to which role instance?
With this information i want to feed the azure.diagnostics.monitor with some performance counters - namely Process(w3wp)\ProcessorTime (%) and Thread Count. But in order to get any meaningfull data i have to append the process ID of the w3wp process to the performance counter (e.g Process(w3wp_PID)\processorTime(%)) - dont know if the syntax is right , but there is a way to append the PID)
so the final result in the AzureStorage table WADPerformanceCounters only has entries like:
WebRoleInstance_n_0 | process(w3wp_1033)\processorTime (%) | 12.4
WebRoleInstance_n_1 | process(w3wp_1055)\processorTime (%) | 48.4
atm its like
WebRoleInstance_n_0 | process(w3wp)\processorTime (%) | 12.4
WebRoleInstance_n_1 | process(w3wp)\processorTime (%) | 12.4
i thought: if i started a DiagnosticsMonitor for each Role, that the monitor would use the corrrect process - belonging to the Roleinstance who started the monitor . but actually that does not work - or i think it doesnt work - at least after looking at the resulting values.
//update: on the manage.windowsazure portal you can define custom metrics for the performance monitoring. It is possible here to chose the webrole instance to be monitored exclusively. This is what i want to do also. Insights on what this page actually does might help also.
for comparison: http://puu.sh/1xp7q
They only stupid way i can think of to get this information is : to get a list of all processes before and after each w3wp is started - identify which one was added and then decide code base context wise which role instance was just started.
i got it working - allthough it was not really straight forward.
first of all i have to make some corrections to my previous statements - just to be on the same level.
In the Cloud Service there are several Virtual Machines, each hosting either a WebRole Instance or a WorkerRole Instance. Thus on a single VM only a single w3wp runs or no w3wp at all but a waworkerhost process.
In my special case there is the possiblity to have two w3wp running on a single VM. so i needed to differenciate between those two - thus requiering me to make some sort of process-Instance association.
What i wanted to log was: The Total CPU Load of a single VM, the CPU Load of the Instance Process running on the VM ( w3wp, waworkerhost).
The PerformanceCounter for Total CPU Load is easy and equal for each VM: \Processor(_Total)\% Processortime for the webrole VM i couldnt just use the \process(w3wp)\% processortime counter because i can not be sure if its the correct w3wp ( see above)
Now here is what i did: Since you have to start a performance counter monitor for each role instance OnStart() in the WebRole.cs or WorkerRole.cs i figured this is the only place where i can somehow gather the required information.
In the WorkerRole.cs i did:
int pc = Environment.ProcessorCount;
string instance = RoleEnvironment.CurrentRoleInstance.Id;
SomeOtherManagementClass.StartDiagnosticMonitorService(pc, instance, Process.GetCurrentProcess());
In the WebRole.cs the CurrentProcess also returns WaWorkerHost, so i had to move the above codelines into the global.asax of the WebRole . Here the correct Process is available.
In the SomeOtherManagementClass i put the StartDiagnosticsMonitorService , which now receives the CurrentProcess from which StartDiagnosticsMonitorService was called. (from workerrole.cs it would receive WaWorkerHost Process and from WebRoles the w3wp process - including PID)
public static void StartDiagnosticMonitorService(int coreCount, string currentRole, Process process)
{
string processName = GetProcessInstanceName(process.Id);
SetCPUCoreData(coreCount, currentRole, processName, process.Id);
...
instanceProcessLoadCounterName = String.Format(@"\Process({0})\% Processor Time", processName);
}
GetProcessInstanceName(process.Id) is now called on each VM and gets the processName to the provided process.Id - this allows you to make a differentiation between multiple w3wps on a single VM because the instanceNames returned are w3wp, w3wp#1, w3wp#2 etc. in contrary to to the processName provided by GetCurrentProcess, which is allways w3wp. for this i modified a codesample i found here on stackoverflow - you can find it below:
private static string GetProcessInstanceName(int pid)
{
PerformanceCounterCategory cat = new PerformanceCounterCategory("Process");
string[] instances = cat.GetInstanceNames();
foreach (string instance in instances)
{
try
{
using (PerformanceCounter cnt = new PerformanceCounter("Process",
"ID Process", instance, true))
{
int val = (int)cnt.RawValue;
if (val == pid)
{
return instance;
}
}
}
catch (InvalidOperationException)
{
//this point is reached when a process terminates while iterating the processlist- this it cannot be found
}
}
return "";
}
Last but not least: SetCPUCoreData(coreCount, currentRole, processName, process.Id) saves all the relevant Data of the processes to the azure storage so it is available from everywhere in the application:
private static void SetCPUCoreData(int count, string roleinstance, string processName, int processID)
{
string[] instances = roleinstance.Split('.');
CloudStorageAccount storageAccount = CloudStorageAccount.Parse(GetSettingValue("LoadMonitor.Connection.String"));
CloudTableClient cloudTableClient = storageAccount.CreateCloudTableClient();
const string tableName = "PerformanceMonitorCoreCount";
cloudTableClient.CreateTableIfNotExist(tableName);
TableServiceContext serviceContext = cloudTableClient.GetDataServiceContext();
PerformanceCounterCPUCoreEntity ent = new PerformanceCounterCPUCoreEntity(count, instances[instances.Count() - 1],processName, processID);
serviceContext.AttachTo(tableName, ent);
serviceContext.UpdateObject(ent);
serviceContext.SaveChangesWithRetries(SaveChangesOptions.ReplaceOnUpdate);
}
the PerformanceCounterCPUCoreEntity is a Template for the StorageTable - look into the azure Storage API if you have any questions regarding this part, or just ask.