mersenneforum.org  

Go Back   mersenneforum.org > Great Internet Mersenne Prime Search > Hardware > Cloud Computing

Reply
 
Thread Tools
Old 2019-09-17, 01:09   #67
Chuck
 
Chuck's Avatar
 
May 2011
Orange Park, FL

2×449 Posts
Default

Quote:
Originally Posted by chalsall View Post
Yeah... I need to handle this exception better...

This happens if you don't have "Internet" turned on (right-hand side; under "Settings"). Note you need to give a cell-phone number to receive an SMS message to activate this. This only has to be done once.
Thanks, it's running now.

I got a Tesla P100 which is about three times faster than the K80 but not as speedy as the T4 on Colab. Are there different GPUs on Kaggle too?
Chuck is online now   Reply With Quote
Old 2019-09-17, 01:18   #68
chalsall
If I May
 
chalsall's Avatar
 
"Chris Halsall"
Sep 2002
Barbados

13·773 Posts
Default

Quote:
Originally Posted by Chuck View Post
Are there different GPUs on Kaggle too?
Nope, only the P100s.

The nice thing about Kaggle is you can do "File" menu -> "Commit" up to twice, to launch batch runs. These run for up to nine (9) hours, even if you close your browser session.

The not-so-nice thing is you're limited to 30 hours of compute per week (not that we have any right to complain...).
chalsall is online now   Reply With Quote
Old 2019-09-17, 01:19   #69
Chuck
 
Chuck's Avatar
 
May 2011
Orange Park, FL

2·449 Posts
Default Checkpoints cross platform?

Are checkpoint files specific to each platform (Colab and Kaggle), or can an abandoned assignment from one platform be picked up and completed on the other?
Chuck is online now   Reply With Quote
Old 2019-09-17, 01:27   #70
chalsall
If I May
 
chalsall's Avatar
 
"Chris Halsall"
Sep 2002
Barbados

274116 Posts
Default

Quote:
Originally Posted by Chuck View Post
Are checkpoint files specific to each platform (Colab and Kaggle), or can an abandoned assignment from one platform be picked up and completed on the other?
Completely cross-compatible.
chalsall is online now   Reply With Quote
Old 2019-09-17, 20:35   #71
Chuck
 
Chuck's Avatar
 
May 2011
Orange Park, FL

11100000102 Posts
Default Kaggle formatting

The output for Kaggle is just a little too long to fit on one line. (It looks like it is the "uptime" data Kaggle is adding to the line is causing this). It causes the lines to wrap about five characters to a second line.
Chuck is online now   Reply With Quote
Old 2019-09-17, 21:17   #72
chalsall
If I May
 
chalsall's Avatar
 
"Chris Halsall"
Sep 2002
Barbados

274116 Posts
Default

Quote:
Originally Posted by Chuck View Post
The output for Kaggle is just a little too long to fit on one line. (It looks like it is the "uptime" data Kaggle is adding to the line is causing this). It causes the lines to wrap about five characters to a second line.
I'm adding the uptime string to each line -- useful metric.

Weird about the wrapping; it's fine on my browser. What screen resolution are you running?

Edit: OK -- easily reproducible: make the window narrower. Is your browser window as wide as your screen will allow? I'm running 1920 pixels wide, and have lots of room before it wraps.

Last fiddled with by chalsall on 2019-09-17 at 21:55
chalsall is online now   Reply With Quote
Old 2019-09-17, 23:51   #73
Dylan14
 
Dylan14's Avatar
 
"Dylan"
Mar 2017

10010010102 Posts
Default

Another suggestion for the bootstrap.pl script: prior to calling

Code:
apt-get -y install libcrypt-ssleay-perl 2>/dev/null
and
Code:
apt-get -y install libwww-perl 2>/dev/null
I would call

Code:
apt-get update
The reason being is that sometimes the Colab environment will fail to get certain packages (especially for packages with lots of dependencies, like boinc). By calling this, the correct links to the repositories will be made before downloading.
By the way, for the Colab, here is a list of all the repositories it uses:


Code:
Hit:1 http://archive.ubuntu.com/ubuntu bionic InRelease 
Get:2 http://archive.ubuntu.com/ubuntu bionic-updates InRelease [88.7 kB] 
Get:3 http://ppa.launchpad.net/graphics-drivers/ppa/ubuntu bionic InRelease [21.3 kB] 
Get:4 http://ppa.launchpad.net/marutter/c2d4u3.5/ubuntu bionic InRelease [15.4 kB] 
Get:5 https://cloud.r-project.org/bin/linux/ubuntu bionic-cran35/ InRelease [3,626 B] 
Get:6 http://archive.ubuntu.com/ubuntu bionic-backports InRelease [74.6 kB] 
Get:7 http://security.ubuntu.com/ubuntu bionic-security InRelease [88.7 kB] 
Ign:8 https://developer.download.nvidia.co...ntu1804/x86_64  InRelease 
Ign:9 https://developer.download.nvidia.co...ntu1804/x86_64  InRelease 
Get:10 https://developer.download.nvidia.co...ntu1804/x86_64  Release [564 B] 
Hit:11 https://developer.download.nvidia.co...ntu1804/x86_64  Release 
Get:12 https://developer.download.nvidia.co...ntu1804/x86_64  Release.gpg [819 B] 
Get:13 http://ppa.launchpad.net/graphics-drivers/ppa/ubuntu bionic/main amd64 Packages [31.5 kB] 
Get:14 http://ppa.launchpad.net/marutter/c2d4u3.5/ubuntu bionic/main Sources [1,686 kB] 
Get:15 http://ppa.launchpad.net/marutter/c2d4u3.5/ubuntu bionic/main amd64 Packages [810 kB] 
Get:16 http://archive.ubuntu.com/ubuntu bionic-updates/restricted amd64 Packages [19.0 kB] 
Get:17 http://archive.ubuntu.com/ubuntu bionic-updates/multiverse amd64 Packages [7,998 B] 
Get:18 http://archive.ubuntu.com/ubuntu bionic-updates/universe amd64 Packages [1,284 kB] 
Get:19 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 Packages [946 kB] 
Get:20 https://cloud.r-project.org/bin/linux/ubuntu bionic-cran35/ Packages [70.5 kB] 
Get:21 http://archive.ubuntu.com/ubuntu bionic-backports/universe amd64 Packages [4,227 B] 
Get:22 http://security.ubuntu.com/ubuntu bionic-security/universe amd64 Packages [766 kB] 
Get:24 https://developer.download.nvidia.co...ntu1804/x86_64  Packages [113 kB] 
Get:25 http://security.ubuntu.com/ubuntu bionic-security/restricted amd64 Packages [8,385 B] 
Get:26 http://security.ubuntu.com/ubuntu bionic-security/main amd64 Packages [649 kB] 
Get:27 http://security.ubuntu.com/ubuntu bionic-security/multiverse amd64 Packages [4,952 B]
Dylan14 is offline   Reply With Quote
Old 2019-09-18, 00:46   #74
chalsall
If I May
 
chalsall's Avatar
 
"Chris Halsall"
Sep 2002
Barbados

13×773 Posts
Default

Quote:
Originally Posted by Dylan14 View Post
Code:
apt-get update
Tried this on Kaggle, and it *really* didn't like it. Thoughts?

Code:
Get:1 http://packages.cloud.google.com/apt cloud-sdk InRelease [6337 B]
Ign:2 http://deb.debian.org/debian stretch InRelease                           
Get:3 http://security.debian.org/debian-security stretch/updates InRelease [94.3 kB]
Err:1 http://packages.cloud.google.com/apt cloud-sdk InRelease
  Couldn't create temporary file /tmp/apt.conf.i71N28 for passing config to apt-key
Err:3 http://security.debian.org/debian-security stretch/updates InRelease
  Couldn't create temporary file /tmp/apt.conf.36xa18 for passing config to apt-key
Get:4 http://deb.debian.org/debian stretch-updates InRelease [91.0 kB]
Err:4 http://deb.debian.org/debian stretch-updates InRelease
  Couldn't create temporary file /tmp/apt.conf.uIGzPc for passing config to apt-key
Get:5 http://deb.debian.org/debian stretch Release [118 kB]
Get:6 http://deb.debian.org/debian stretch Release.gpg [2365 B]
Err:6 http://deb.debian.org/debian stretch Release.gpg
  Couldn't create temporary file /tmp/apt.conf.xhjRLe for passing config to apt-key
Fetched 2365 B in 0s (19.4 kB/s)
Reading package lists... Done
W: An error occurred during the signature verification. The repository is not updated and the previous index files will be used. GPG error: http://packages.cloud.google.com/apt cloud-sdk InRelease: Couldn't create temporary file /tmp/apt.conf.i71N28 for passing config to apt-key
W: An error occurred during the signature verification. The repository is not updated and the previous index files will be used. GPG error: http://security.debian.org/debian-security stretch/updates InRelease: Couldn't create temporary file /tmp/apt.conf.36xa18 for passing config to apt-key
W: An error occurred during the signature verification. The repository is not updated and the previous index files will be used. GPG error: http://deb.debian.org/debian stretch-updates InRelease: Couldn't create temporary file /tmp/apt.conf.uIGzPc for passing config to apt-key
W: An error occurred during the signature verification. The repository is not updated and the previous index files will be used. GPG error: http://deb.debian.org/debian stretch Release: Couldn't create temporary file /tmp/apt.conf.xhjRLe for passing config to apt-key
W: Failed to fetch http://security.debian.org/debian-security/dists/stretch/updates/InRelease  Couldn't create temporary file /tmp/apt.conf.36xa18 for passing config to apt-key
W: Failed to fetch http://deb.debian.org/debian/dists/stretch-updates/InRelease  Couldn't create temporary file /tmp/apt.conf.uIGzPc for passing config to apt-key
W: Failed to fetch http://packages.cloud.google.com/apt/dists/cloud-sdk/InRelease  Couldn't create temporary file /tmp/apt.conf.i71N28 for passing config to apt-key
W: Failed to fetch http://deb.debian.org/debian/dists/stretch/Release.gpg  Couldn't create temporary file /tmp/apt.conf.xhjRLe for passing config to apt-key
W: Some index files failed to download. They have been ignored, or old ones used instead.
chalsall is online now   Reply With Quote
Old 2019-09-18, 00:54   #75
Chuck
 
Chuck's Avatar
 
May 2011
Orange Park, FL

2×449 Posts
Default Kaggle wide output

I have just observed what is causing the "wide" Kaggle output. When the uptime goes beyond "23:59", it starts outputting "1 day, 4 min" etc. These additional characters are causing the line wrap.
Chuck is online now   Reply With Quote
Old 2019-09-18, 01:04   #76
Chuck
 
Chuck's Avatar
 
May 2011
Orange Park, FL

2×449 Posts
Default Kaggle over 1 day

Quote:
Originally Posted by Chuck View Post
I have just observed what is causing the "wide" Kaggle output. When the uptime goes beyond "23:59", it starts outputting "1 day, 4 min" etc. These additional characters are causing the line wrap.
Code:
20190918_005351 INFO:  69453401  74 to 75    0.8%    31/4620,   8/960 | 1173.47 |   4.225s |  1h07m |  29.43G | 6966.7M/s |   82485 | 1 day, 4 min
20190918_005453 INFO:  69453401  74 to 75    2.4%   103/4620,  23/960 | 1168.49 |   4.243s |  1h06m |  29.43G | 6937.2M/s |   82485 | 1 day, 5 min
20190918_005556 INFO:  69453401  74 to 75    4.0%   168/4620,  38/960 | 1171.81 |   4.231s |  1h05m |  29.43G | 6956.8M/s |   82485 | 1 day, 6 min
20190918_005656 INFO:  69453401  74 to 75    5.4%   243/4620,  52/960 | 1170.15 |   4.237s |  1h04m |  29.43G | 6947.0M/s |   82485 | 1 day, 7 min
20190918_005800 INFO:  69453401  74 to 75    7.0%   304/4620,  67/960 | 1168.22 |   4.244s |  1h03m |  29.43G | 6935.5M/s |   82485 | 1 day, 8 min
20190918_005903 INFO:  69453401  74 to 75    8.5%   376/4620,  82/960 | 1169.87 |   4.238s |  1h02m |  29.43G | 6945.3M/s |   82485 | 1 day, 9 min
20190918_010003 INFO:  69453401  74 to 75   10.0%   448/4620,  96/960 | 1170.70 |   4.235s |  1h00m |  29.43G | 6950.3M/s |   82485 | 1 day, 10 min
20190918_010106 INFO:  69453401  74 to 75   11.6%   523/4620, 111/960 | 1164.38 |   4.258s |  1h00m |  29.43G | 6912.7M/s |   82485 | 1 day, 11 min
20190918_010206 INFO:  69453401  74 to 75   13.0%   588/4620, 125/960 | 1169.59 |   4.239s | 59m00s |  29.43G | 6943.7M/s |   82485 | 1 day, 12 min
Here is some of it
Chuck is online now   Reply With Quote
Old 2019-09-18, 01:08   #77
Dylan14
 
Dylan14's Avatar
 
"Dylan"
Mar 2017

2×293 Posts
Default

Quote:
Originally Posted by chalsall View Post
Tried this on Kaggle, and it *really* didn't like it. Thoughts?

Code:
Get:1 http://packages.cloud.google.com/apt cloud-sdk InRelease [6337 B]
Ign:2 http://deb.debian.org/debian stretch InRelease                           
Get:3 http://security.debian.org/debian-security stretch/updates InRelease [94.3 kB]
Err:1 http://packages.cloud.google.com/apt cloud-sdk InRelease
  Couldn't create temporary file /tmp/apt.conf.i71N28 for passing config to apt-key
Err:3 http://security.debian.org/debian-security stretch/updates InRelease
  Couldn't create temporary file /tmp/apt.conf.36xa18 for passing config to apt-key
Get:4 http://deb.debian.org/debian stretch-updates InRelease [91.0 kB]
Err:4 http://deb.debian.org/debian stretch-updates InRelease
  Couldn't create temporary file /tmp/apt.conf.uIGzPc for passing config to apt-key
Get:5 http://deb.debian.org/debian stretch Release [118 kB]
Get:6 http://deb.debian.org/debian stretch Release.gpg [2365 B]
Err:6 http://deb.debian.org/debian stretch Release.gpg
  Couldn't create temporary file /tmp/apt.conf.xhjRLe for passing config to apt-key
Fetched 2365 B in 0s (19.4 kB/s)
Reading package lists... Done
W: An error occurred during the signature verification. The repository is not updated and the previous index files will be used. GPG error: http://packages.cloud.google.com/apt cloud-sdk InRelease: Couldn't create temporary file /tmp/apt.conf.i71N28 for passing config to apt-key
W: An error occurred during the signature verification. The repository is not updated and the previous index files will be used. GPG error: http://security.debian.org/debian-security stretch/updates InRelease: Couldn't create temporary file /tmp/apt.conf.36xa18 for passing config to apt-key
W: An error occurred during the signature verification. The repository is not updated and the previous index files will be used. GPG error: http://deb.debian.org/debian stretch-updates InRelease: Couldn't create temporary file /tmp/apt.conf.uIGzPc for passing config to apt-key
W: An error occurred during the signature verification. The repository is not updated and the previous index files will be used. GPG error: http://deb.debian.org/debian stretch Release: Couldn't create temporary file /tmp/apt.conf.xhjRLe for passing config to apt-key
W: Failed to fetch http://security.debian.org/debian-security/dists/stretch/updates/InRelease  Couldn't create temporary file /tmp/apt.conf.36xa18 for passing config to apt-key
W: Failed to fetch http://deb.debian.org/debian/dists/stretch-updates/InRelease  Couldn't create temporary file /tmp/apt.conf.uIGzPc for passing config to apt-key
W: Failed to fetch http://packages.cloud.google.com/apt/dists/cloud-sdk/InRelease  Couldn't create temporary file /tmp/apt.conf.i71N28 for passing config to apt-key
W: Failed to fetch http://deb.debian.org/debian/dists/stretch/Release.gpg  Couldn't create temporary file /tmp/apt.conf.xhjRLe for passing config to apt-key
W: Some index files failed to download. They have been ignored, or old ones used instead.

This error would suggest that perhaps / is read only on Kaggle (as per the replies here: https://www.linuxquestions.org/quest...es-4175619721/). But that doesn't make sense, since we are able to write to the disk to run the bootstrap script.
Also, per the man page for apt-get there doesn't seem to be an option to change the location where the temp files are written to.

Last fiddled with by Dylan14 on 2019-09-18 at 01:11 Reason: adding apt-get man page
Dylan14 is offline   Reply With Quote
Reply

Thread Tools


Similar Threads
Thread Thread Starter Forum Replies Last Post
Alternatives to Google Colab kriesel Cloud Computing 11 2020-01-14 18:45
Notebook enzocreti enzocreti 0 2019-02-15 08:20
Computer Diet causes Machine Check Exception -- need heuristics help Christenson Hardware 32 2011-12-25 08:17
Computer diet - Need help garo Hardware 41 2011-10-06 04:06
Workunit diet ? dsouza123 NFSNET Discussion 5 2004-02-27 00:42

All times are UTC. The time now is 17:56.


Mon Dec 6 17:56:15 UTC 2021 up 136 days, 12:25, 0 users, load averages: 1.82, 1.92, 1.82

Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2021, Jelsoft Enterprises Ltd.

This forum has received and complied with 0 (zero) government requests for information.

Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License, Version 1.2 or any later version published by the Free Software Foundation.
A copy of the license is included in the FAQ.