Genvid Forum

Services job won't start


#1

Hi,

My cloud services job, which worked fine earlier today, is now refusing to start, without me having changed any settings etc. (other than my development PC sleeping/disconnecting/reconnecting). On attempting to start the services job, I get a long log of repeating errors, that look like this:

Error applying template: 2018/05/03 14:52:36.328841 [DEBUG] (runner) receiving dependency catalog.datacenters

2018/05/03 14:52:36.329795 [DEBUG] (runner) kv.block(genvid/images/rootd/url) is still needed
2018/05/03 14:52:36.329795 [DEBUG] (runner) kv.block(genvid/images/gvencode/url) is still needed
2018/05/03 14:52:36.329795 [DEBUG] (runner) kv.block(genvid/images/eventsd/dirty) is still needed
2018/05/03 14:52:36.329795 [DEBUG] (runner) kv.block(genvid/images/eventsd/url) is still needed
2018/05/03 14:52:36.329795 [DEBUG] (runner) kv.block(genvid/images/rootd/tag) is still needed
2018/05/03 14:52:36.329795 [DEBUG] (runner) kv.block(genvid/images/rootd/image) is still needed
2018/05/03 14:52:36.329795 [DEBUG] (runner) kv.block(genvid/images/eventsd/image) is still needed
2018/05/03 14:52:36.329795 [DEBUG] (runner) vault.read(secret/disco) is still needed
2018/05/03 14:52:36.329795 [DEBUG] (runner) kv.block(genvid/images/discod/url) is still needed
2018/05/03 14:52:36.329795 [DEBUG] (runner) kv.block(genvid/images/composed/image) is still needed
2018/05/03 14:52:36.329795 [DEBUG] (runner) kv.block(genvid/images/leafd/url) is still needed
2018/05/03 14:52:36.329795 [DEBUG] (runner) kv.block(genvid/encode/stream/key) is still needed
2018/05/03 14:52:36.329795 [DEBUG] (runner) kv.block(genvid/images/composed/url) is still needed
2018/05/03 14:52:36.329795 [DEBUG] (runner) catalog.datacenters is still needed
2018/05/03 14:52:36.329795 [DEBUG] (runner) kv.block(genvid/images/discod/image) is still needed
2018/05/03 14:52:36.329795 [DEBUG] (runner) kv.block(genvid/images/leafd/dirty) is still needed
2018/05/03 14:52:36.329795 [DEBUG] (runner) vault.read(secret/command) is still needed
2018/05/03 14:52:36.329795 [DEBUG] (runner) kv.block(genvid/images/eventsd/tag) is still needed
2018/05/03 14:52:36.329795 [DEBUG] (runner) kv.block(genvid/images/discod/tag) is still needed
2018/05/03 14:52:36.329795 [DEBUG] (runner) kv.block(genvid/images/leafd/tag) is still needed
2018/05/03 14:52:36.329795 [DEBUG] (runner) kv.block(genvid/images/commandd/dirty) is still needed
2018/05/03 14:52:36.329795 [DEBUG] (runner) kv.block(genvid/images/gvencode/tag) is still needed
2018/05/03 14:52:36.329795 [DEBUG] (runner) kv.block(genvid/images/gvencode/image) is still needed
2018/05/03 14:52:36.329795 [DEBUG] (runner) kv.block(genvid/images/composed/dirty) is still needed
2018/05/03 14:52:36.329795 [DEBUG] (runner) kv.block(genvid/images/commandd/tag) is still needed
2018/05/03 14:52:36.329795 [DEBUG] (runner) kv.block(genvid/images/commandd/url) is still needed
2018/05/03 14:52:36.329795 [DEBUG] (runner) kv.block(genvid/encode/stream/addr) is still needed
2018/05/03 14:52:36.329795 [DEBUG] (runner) kv.block(genvid/images/rootd/dirty) is still needed
2018/05/03 14:52:36.329795 [DEBUG] (runner) kv.block(genvid/images/commandd/image) is still needed
2018/05/03 14:52:36.329795 [DEBUG] (runner) kv.block(genvid/images/composed/tag) is still needed
2018/05/03 14:52:36.329795 [DEBUG] (runner) kv.block(genvid/images/discod/dirty) is still needed
2018/05/03 14:52:36.329795 [DEBUG] (runner) kv.block(genvid/images/leafd/image) is still needed
2018/05/03 14:52:36.329795 [DEBUG] (runner) kv.block(genvid/images/gvencode/dirty) is still needed

2018/05/03 14:52:36.329795 [DEBUG] (runner) receiving dependency kv.block(genvid/images/commandd/tag)


#2

Hi Adrian,

Did you first run the following commands before attempting to start the services?
genvid-sdk -c mycluster upload-images-sdk -u
genvid-sdk -c mycluster setup
genvid-sdk -c mycluster load-config-sdk
(replacing “mycluster” by your custom cluster name)

Thanks,
Sophie


#3

Hi Sophie,

I did run those commands when the jobs were first created, and it worked the first time.

I then just stopped the job and tried to start it again (a process which works on a local cluster), does it require those commands to be run each time the jobs are stopped/started ?

I tried it out and it seems just running the genvid-sdk -c mycluster setup command is enough to make the error go away (there doesn’t seem to be a need to upload the images again as they are already on S3?) - and I can now start and stop the job without having to re-run it.

However, I suspect the issue may occur again (it’s like it had got itself into some kind of cycle of being broken), will post back if it does.

Thanks,

Adrian


#4

Hi Adrian,

I’m glad that setup was sufficient to solve the issue for now, but it is rather strange behavior nonetheless. You do not need to upload-images or load-config every time, I was just trying to ensure it had already been done.

Keep me posted!

Cheers,
Sophie