Colab question

[kladner]

Originally Posted by LOBES

My account is paid up until November 29. If things improve to how they used to be (24hrs per single GPU session) then I'll renew. If not, I'll let it expire and just do a freebie account.

I feel bad for chalsall, as I imagine throughput at GPU72 will go down a little.

I am feeling that I should follow your path and set a time limit on paid. I almost always get 12 hours on free CPU runs, for at least one of two workers on an account. I don't see an even split with 2 running on a account. One will run to 12 hours and the other will stop somewhere a lot sooner.

Delayed Edit: I did cancel the paid accounts. While I still have them, I seem to be able to get 24 hour CPU runs, sometimes. Responses for Paid and Free notebooks are unpredictable. Forget about GPU runs of any useful length. You may get a couple of hours of T4 for free. Paid V100 times out in minutes.

On Paid, I have encountered "Cannot connect to GPU backend due to blah blah blah." It's more common on Free to get "Too many sessions" for the second notebook on an account. This is unpredictable. I have 4 free accounts, with 2 worker each running right now. They are all in the1-3 hour range, with a majority 3-ish. I have one paid CPU closing in on 24 hours.

When I can get 12 hour CPU runs from free accounts, I see no point in Paid. Without high end GPUs it's not worth it. I have one stranded TF assignment. I'm not in a position to finish it without snagging another assignment. That is, I have no sense of manual control. I just run the beasties.

[LOBES]

My free accounts are out-performing my paid account.

That is all that needs to be said. Next month my $9.99 will go toward a Taco Bell feast. Because THAT is a gift that will keep on giving.....if you know what I mean.

[Aramis Wyler]

I don't even know what they could be thinking, chopping down the runtimes like this. There are no notices or anything that I could find, and it's just rediculous. I can still run 4 sessions, but they only go for a few minutes, and 1 session does not run much longer if I run it alone.

[Runtime Error]

Has anyone else noticed variability in throughput on the colab hardware? For same range exponents, I've seen everywhere from high 1200s to low 1700 ghz/d/d on Tesla T4s for trial factoring. Most commonly is ~1600, but these discrepancies go beyond variation explainable by the "silicon lottery". I'm curious if anyone has a conjecture on what they might be doing with their hardware. Thanks!

[moebius]

Originally Posted by Runtime Error

Has anyone else noticed variability in throughput on the colab hardware?

I can only say that the values ​​for LL tests with gpuowl 6.11.380, in the 55M range,deviate for Tesla T4's from 3000 us/it to 3400 us/it, whereas with the P100's they only differ by a few us/it. That doesn't quite explain the high variabilityst TF in throughput. I always get Tesla T4's, so I just pick a quick one for LL-DC's.

[chalsall]

Originally Posted by Runtime Error

Has anyone else noticed variability in throughput on the colab hardware?

Personally, I've seen nothing like what you describe.

I only accept T4s nowadays (even a P100 is not acceptable, and it is thrown back into the pool, to be quickly replaced with a T4 during the fishing exercise).

Extreme weirdness.

It's almost as if Google's Colab is playing with each of us individually... Non-zero probability that speculation is actualy true...

[kladner]

I have seen a 9-10% variation in the reported speeds of V100 GPUs. My paid accounts are winding down, but lately they have seemed to offer far more P100s than v100s.
My frustration with free T4s is the short runtimes. Otherwise, I see them as one of the best cards out there. In rough terms, they deliver ~50% more than a P100, and about half of a V100. To me, the big deal is that T4 does this on 72W instead of the 300W drawn by the (X)100s.

[LOBES]

My paid account is at least getting 24 hour run times, with non-GPU only. Looks like the remainder of my time will be doing P-1 until the subscription runs out.

[mrk74]

(I wasn't sure whether to put this here or under "GPU to 72 status" so move it as needed)


Is it just me or are TF's just not coming up? I do have it set to do what's needed so maybe it's just that but I haven't seen a TF in probably 3 days.

[Flaukrotist]

Originally Posted by mrk74

Is it just me or are TF's just not coming up? I do have it set to do what's needed so maybe it's just that but I haven't seen a TF in probably 3 days.

I guess, you mean GPUs. I haven't seen any GPU for my two free colab accounts for 7 and 9 days, respectively. Before it was consistently 2-3 hours of T4 (for TF) per day. So they seem to have changed their allocation policy.

What's even more, I have the feeling that also the 12 hours of CPU get stopped after less than one hour frequently since the absence of the GPUs and cannot be reconnected and continued like before.

[mrk74]

Originally Posted by Flaukrotist

I guess, you mean GPUs. I haven't seen any GPU for my two free colab accounts for 7 and 9 days, respectively. Before it was consistently 2-3 hours of T4 (for TF) per day. So they seem to have changed their allocation policy.

What's even more, I have the feeling that also the 12 hours of CPU get stopped after less than one hour frequently since the absence of the GPUs and cannot be reconnected and continued like before.

Right a GPU. I'm going on about 10 days. Would it make a difference to switch the setting to TPU since we aren't getting a GPU anyway?