Google Diet Colab Notebook

[xx005fs]

Is there anyway to run 1 session but execute 2 programs at once, say executing primenet.py and GPUOWL to manage assignments or MISFIT and mfactx? or is there a need to open up a second session to run the assignment managers and will those affect my GPU session run time?

[chalsall]

Originally Posted by xx005fs

Is there anyway to run 1 session but execute 2 programs at once, say executing primenet.py and GPUOWL to manage assignments or MISFIT and mfactx?

Well, MISFIT is going to be a problem, since it's Winblows only...

But to answer your question, sure. Just launch it (or anything(s)) you want into the background.

But... be aware that everything launched (even when fork()'ed) is terminated when the Section finishes running, so you need to keep the launching script running (possibly just sleep(forever)).

[kriesel]

Originally Posted by xx005fs

Is there any way to run 1 session but execute 2 programs at once

Yes. Or 3 or more.

Launch task A as a background task, go on to launch task B as a background task, and task C showing periodic top updates is my normal MO in each colab session. The colab cpus are fast enough that if you're a frequent colab user, a 90M primality test will complete in time before assignment expiration.

It's linux on a VM, so multitasking is built in, no need for separate Colab sessions for each task. My task A is mprime to occupy the cpu core; B a gpu app if I can get a gpu, otherwise omitted; C is top repeating at 2-minute intervals in the foreground, that shows uptime and whether mprime and the gpu app are running and if so how much cpu time each is getting. That gives pretty good resolution of how long a session lasted.

Another background task could be wedged in after the launch of task B to run on cpu also such as a primenet.py or other helper app. Or an additional gpu task on the same allocated gpu, to get the small total throughput gains that are common even in well tuned TF applications on the faster gpus. (Commonly 1-3% boost with a second instance.)

If the gpu times out, the session times out.

If the gpu runs out of work, the gpu app and its background process terminate, while mprime keeps going until cpu use times out.

Top regularly repeating (!top -d 120 or however many seconds interval you like, as long as it's shorter than Colab's idle-session detection) keeps the session going.

By all means run mprime along with the gpu app; don't leave the cpu idle/wasted.

Do LL DC if nothing else; there's a several year backlog.
Sample code for multiple-task sections has been posted.

Follow the link in https://mersenneforum.org/showpost.p...&postcount=891 or see https://www.mersenneforum.org/showpo...73&postcount=8 etc.

, say executing primenet.py and GPUOWL to manage assignments or MISFIT and mfactx? or is there a need to open up a second session to run the assignment managers and will those affect my GPU session run time?

I have no running experience on MISFIT. Getting it to run on colab might be a challenge; MS Forms & .net required. If I recall correctly there is also a graphical interface. I've seen claims it may be runnable on linux with mono. Never tried it myself.

There are alternatives. See https://www.mersenneforum.org/showpo...92&postcount=3
Another possibility is to add softare on your pc to make a Google drive appear local, and run MISFIT or another management app locally.
In my work mix, running TF to 76 bits usually, the work addition and results reporting only needs to be daily or less, hardly worth scripting into colab. It would be different at 73 bits or even lower.

In general, for reference info, see https://mersenneforum.org/showthread.php?t=24607

[kriesel]

I thought a Colab session only got one cpu core. But I notice today it looks like 2, with HT.

Architecture:        x86_64 CPU 
op-mode(s):      32-bit, 64-bit 
Byte Order:          Little Endian 
CPU(s):              2 
On-line CPU(s) list: 0,1 
Thread(s) per core:  2 
Core(s) per socket:  1 
Socket(s):           1 
NUMA node(s):        1 
Vendor ID:           GenuineIntel 
CPU family:          6 Model:               79 
Model name:          Intel(R) Xeon(R) CPU @ 2.20GHz

[axn]

Originally Posted by kriesel

I thought a Colab session only got one cpu core. But I notice today it looks like 2, with HT.

Not sure whether you think 2 cores/4 threads or 1 core/2 threads, but it is in fact the latter.
For P95 use, it is just one core.

[chalsall]

Originally Posted by Chuck

Finally. I've signed up. Let's see how this works out.

Well, things have /definitely/ changed on Colab. For the better; at least at the moment.

A massive amount of compute is being given. Personally, after almost a month getting effectively nothing, I'm at ~3.5 THzD/D for the last five days.

Thanks, Google!!!

Chuck... How are you finding the "paid tier"? I'd try it myself, but I don't have any US-based financial routes.

[chalsall]

Originally Posted by chalsall

Personally, after almost a month getting effectively nothing, I'm at ~3.5 THzD/D for the last five days.

A new personal record... Six (6#) T4s running simultaneously!!!

And over-night, one T4 lasted 11 hours, while three others lasted 10 hours each!

[kriesel]

Originally Posted by chalsall

A new personal record... Six (6#) T4s running simultaneously!!!

And over-night, one T4 lasted 11 hours, while three others lasted 10 hours each!

So that's where they've all gone.

[Chuck]

Originally Posted by chalsall

Chuck... How are you finding the "paid tier"? I'd try it myself, but I don't have any US-based financial routes.

It's working well so far. It disconnects after 24 hours. I had a T4 the first session and P100 thereafter.

[xx005fs]

After the colab premium launch, it has been a complete disaster for me since the only GPU I am receiving is the Tesla T4s, and I can't even get a half decent GPU like the K80 to run my PRP tasks. Even after 30 session resets I am still getting useless T4s, which I would much rather hand it to someone else doing TF since I am not going to run TF workloads. It would've been much better if google decided to leave the K80 GPUs for free users to tinker with.

[axn]

Originally Posted by xx005fs

After the colab premium launch, it has been a complete disaster for me since the only GPU I am receiving is the Tesla T4s, and I can't even get a half decent GPU like the K80 to run my PRP tasks. Even after 30 session resets I am still getting useless T4s, which I would much rather hand it to someone else doing TF since I am not going to run TF workloads. It would've been much better if google decided to leave the K80 GPUs for free users to tinker with.

In my timing runs, a T4 was only about 15% slower than K80 (since you get only half a K80 in colab).

Try this - when you get a T4, use it run your PRP. After 1-2 hours, kill the run and ask for a new GPU session. There is a high chance you will get a P100 or a K80.