"Unable to connect to the runtime" - Colab

[mrk74]

Does anybody know what the usage limits for Colab are? I've maybe let it go for 12 hrs a day for a few days. Haven't been able to connect to a GPU or TPU for about 14-15 hrs because of usage limits. All I get is "Unable to connect to the runtime." Would getting another key help or is it connected to the login? ANY help in understand is greatly appreciated!

[kriesel]

moderator please move this thread to the cloud computing subforum's Colab area. It's Cola-specific, unrelated to PrimeNet.

[chalsall]

Originally Posted by mrk74

Does anybody know what the usage limits for Colab are? I've maybe let it go for 12 hrs a day for a few days. Haven't been able to connect to a GPU or TPU for about 14-15 hrs because of usage limits. All I get is "Unable to connect to the runtime."!

This is nominal behavior.

No one except Google (and perhaps not even their humans) know exactly what the usage limits are. They seem to change over time -- sometimes suddenly, sometimes gradually.

Here's my empirical: after the initial "honeymoon" of a Google (read: Gmail) ***account*** has started using Colab, the availability of a GPU tends to converge to get a single instance run once per day. Lately, they've been running about 7 to 10 hours, tending to the lower range.

Also, when the GPU availability window "opens" seems to be the same time each day. Some of my accounts get GPUs at around 1400 UTC, others not until around 2200. They are always offered (and get) a CPU instance if the GPU isn't available.

Originally Posted by mrk74

Would getting another key help or is it connected to the login? ANY help in understand is greatly appreciated!

Completely tied to the Gmail login. But Google doesn't seem to care how many accounts you use. I'm running eight (across three "humans" (read: VPNs)), and I'm on the free tier. I know at least one person on the paid tier running four concurrently, often almost 24/7.

Lastly, I've observed that new Colab users (even by a newly created Gmail account) initially gets about 12 hour runs of T4s or P100s, which can often immediately be relaunched. This lasts for two or three days, and then the usage is constrained as above.

May we live in interesting times...

[mrk74]

Originally Posted by chalsall

This is nominal behavior.

No one except Google (and perhaps not even their humans) know exactly what the usage limits are. They seem to change over time -- sometimes suddenly, sometimes gradually.

Here's my empirical: after the initial "honeymoon" of a Google (read: Gmail) ***account*** has started using Colab, the availability of a GPU tends to converge to get a single instance run once per day. Lately, they've been running about 7 to 10 hours, tending to the lower range.

Also, when the GPU availability window "opens" seems to be the same time each day. Some of my accounts get GPUs at around 1400 UTC, others not until around 2200. They are always offered (and get) a CPU instance if the GPU isn't available.



Completely tied to the Gmail login. But Google doesn't seem to care how many accounts you use. I'm running eight (across three "humans" (read: VPNs)), and I'm on the free tier. I know at least one person on the paid tier running four concurrently, often almost 24/7.

Lastly, I've observed that new Colab users (even by a newly created Gmail account) initially gets about 12 hour runs of T4s or P100s, which can often immediately be relaunched. This lasts for two or three days, and then the usage is constrained as above.

May we live in interesting times...

If I'm being honest I have no idea what T4 or P100 means but thanks for the info! I guess I'll just have to keep trying till I can latch on.

[Uncwilly]

Originally Posted by chalsall

Lately, they've been running about 7 to 10 hours, tending to the lower range.

I have been getting ~6-8 hours recently. But, using your tip of using the "Factor Reset" option, I fish until I get a P100 or T4 before holding on to them. Also, if you miss the start of your 24 hour reset, it seems that when you do restart, you may reset your 24 hour window.

[pepi37]

Originally Posted by mrk74

If I'm being honest I have no idea what T4 or P100 means but thanks for the info! I guess I'll just have to keep trying till I can latch on.

That is Nvidia GPU, very powerful and expensive card. And very fast...

[Chuck]

Originally Posted by chalsall

I know at least one person on the paid tier running four concurrently, often almost 24/7.

May we live in interesting times...

For a time on the paid tier I was getting 24 hour sessions. Lately they have been limited to 18 hours.

[chalsall]

Interesting...

So, just now I noticed that /all/ of my sessions stopped. Three GPUs and five CPU only.

Then I noticed on one of my Admin reports on GPU72 that *every* GPU72_TF Notebook user also suddenly stopped reporting work underway. This appears to have happened at around 1619 UTC.

I was able to relaunch all my instances. Those that were previously running GPUs were again allowed to get them -- two T4s and a P100.

I'm inferring that Google did some sort of an upgrade, and had to restart everything to accomplish this. So, anyone at console, you might want to try reattaching and restarting your sessions.

[PhilF]

Originally Posted by chalsall

Interesting...

So, just now I noticed that /all/ of my sessions stopped. Three GPUs and five CPU only.

Then I noticed on one of my Admin reports on GPU72 that *every* GPU72_TF Notebook user also suddenly stopped reporting work underway.

I bet you were a bit apprehensive at that point, huh?

[kuratkull]

CPU instances (for LLR64) have been stable/predictable for the last couple of weeks. I use two accounts with 4 instances each. They both get about 12 hours of runtime a day. Both accounts expire and become available again at roughly the same times.

[linament]

For the first time, I received this message on Colab when attempting to run without a GPU: "Sorry, no backends available. Please try again later."