Genvid Forum

Consul - High CPU Usage


#1

Hello,

when running my own project I sometimes get very weird spikes of high CPU-usage by consul. Restarting the services usually fixes the problem.

Consul is at 0.0%-0.5% CPU usage for most of the time, but suddenly it spikes up to 13%-34% (which is more than one core).

I have no real clue when this happens. This happens irregardless of me actually submitting video data or not, however I do strongly believe the stress is caused by gvencode.exe. Once I sent even a single frame consul and gvencode can go crazy, and remain that way, even if I stop submitting video data.

  • Restarting services fixes the problem
  • gvencode.exe is the only other thread consuming major CPU (same range as consul), sometimes composed as well.
  • killing gvencode either via the task-manager or the Cluster-UI stops counsul’s high CPU usage, this behaviour is not there for any other of the services (I tried).

I haven’t found a reliable way to trigger the high CPU usage, except “let it run, after 30 minutes it’s always there”.

I am not entirely sure what additional info I can provide. I’m fairly certain that I don’t do anything crazy but using the C#-SDK.

The problem occurs when submitting video-data as a byte[], and as a D3D11Texture2D.

I also noticed that the version of consul in the SDK is almost a year old now:

c:\Genvid\Genvid Technologies SDK 1.11.0\bin>consul.exe version
Consul v0.7.5

However, I doubt that’s the problem, and I do understand how upgrading it might be a major undertaking.


#2

Hi Moritz,

We will log this in our database and investigate. Thank you for the detailed report, this will greatly help us in resolving the issue.

Thanks,
Sophie


#3

To give more news:
We were able to reproduce the behaviour and true, it seems related to consul behaviour when some check are blocking or behave badly. We are currently fixing such check and planned to upgrade consul as well very soon (especially that 1.0 is out!).
The fixed checks will be in the next version, but for consul, it will very likely wait a bit longer: the new consul have a different configuration format and we are in the process of rewriting the way we handle our background services configuration (consul, nomad and vault) to be more flexible and user friendly.


#4

The (admittedly bad) workaround that we have found is simply setting the process-priority of consul to “low”, this doesn’t seem to hamper genvid that much, that seems to prevent dropped frames for the most part.

I’m looking forward to the next version then :slight_smile: