How I Create a Colab Session to Run a CADO-NFS Client for a Remote Server

[EdH]

(Note: I expect to keep the first post of each of these "How I..." threads up-to-date with the latest version. Please read the rest of each thread to see what may have led to the current set of instructions.)

I will take the liberty of expecting readers to already be somewhat familiar with Google's Colaboratory sessions. There are several threads already on Colab and these should be reviewed by interested readers:

Google Colaboratory Notebook?
GPU72 Notebook Integration...
Notebook Instance Reverse SSH and HTTP Tunnels.
Colab question

I do not, as of yet, have a github account, so I have not created an upload of this to github. Others may feel free to do so, if desired.

The following is intended to allow a user to create and operate a CADO-NFS client to perform polyselect and/or Sieving for a remote server. The server must accept connections from this Colab based client.

The user will need to obtain credentials from the server operator. These credentials are specifically, the web address and the sha1 certificate value.

In order for this to run, the first time it is invoked, it takes about twenty minutes to retrieve and build CADO-NFS before the WUs are obtained and processed. Subsequent runs will start immediately.

On to the specifics:

Open a Google Colaboratory session.
Sign in with your Google/Gmail account info.
Choose New Python3 notebook:

Menu->File->New notebook

Click Connect to start a session.
Edit title from Untitled... to whatever you like.
Paste the following into the Codeblock:

####################################################
### This Colaboratory session is designed to run ###
### a CADO-NFS remote client.                    ###
####################################################

# Set these values (Obtain server and cert from whomever is running the server.)
server = "<full server address>" # remove < and >, but leave quotes
certificate = "<sha1>" # remove < and >, but leave quotes
client = "Colab." # add to or edit this for uniqueness
single = "yes" # Choose whether to run a single Workunit only

import os
import shutil
import socket
import subprocess
import sys
import time

# This section determines whether CADO-NFS is already installed.
# If CADO-NFS does not yet exist, it is built. The initial
# building of CADO-NFS takes about twenty minutes.
exists = os.path.isfile('/content/cado-nfs/README.msieve')
if exists < 1:
  fstart = time.time()
  print("Updating and installing system files (libgmp-dev, cmake, git). . .")
  subprocess.call(["apt", "update"])
  subprocess.call(["apt", "install", "libgmp-dev", "cmake", "git"])
  print("Retrieving CADO-NFS. . .")
  subprocess.call(["git", "clone", "https://gitlab.inria.fr/cado-nfs/cado-nfs.git"])
  os.chdir("/content/cado-nfs")
  print("Building CADO-NFS. This will take about twenty minutes. . .")
  p = subprocess.Popen(["make"], stdout=subprocess.PIPE, stderr=subprocess.STDOUT)
  for line in iter(p.stdout.readline, b''):
    print(line.rstrip().decode("utf=8"))
  runtime = time.time() - fstart
  print("Elapsed time was", int(runtime / 60), "minutes and", int(runtime % 60), "seconds.\n")

# Main client invocation
os.chdir("/content/cado-nfs")
hostname = socket.gethostname()
if client == "Colab.":
  client = client + str(int(time.time()))
client = "--client=" + client
server = "--server=" + server
certificate = "--certsha1=" + certificate
bindir = "--bindir=/content/cado-nfs/build/" + hostname + "/"
extra = "--nocncheck"
fstart = time.time()
if single == "yes":
  p = subprocess.Popen(["./cado-nfs-client.py", client, bindir, server, certificate, extra, "--single", "--override", "t", "2"], stdout=subprocess.PIPE, stderr=subprocess.STDOUT)
else:
  p = subprocess.Popen(["./cado-nfs-client.py", client, bindir, server, certificate, extra, "--override", "t", "2"], stdout=subprocess.PIPE, stderr=subprocess.STDOUT)
for line in iter(p.stdout.readline, b''):
  print(line.rstrip().decode("utf=8"))
if single == "yes":
  runtime = time.time() - fstart
  print("Elapsed time was", int(runtime / 60), "minutes and", int(runtime % 60), "seconds.\n")

To run the process, you will need the server address and the certificate value and edit them into the appropriate location near the top of the script. You should edit the client identification, also. The script is currently set to run a single WU and quit. You should test with this setting before turning the process loose.

After all the editing, you should be ready to run it by clicking the icon or pressing CTRL-Enter. If all is working properly, a message, "Subprocess has PID ####" will eventually appear after the building of CADO-NFS and the downloading of the prerequisite files.