View Single Post
Old 2021-02-03, 19:04   #1
EdH
 
EdH's Avatar
 
"Ed Hall"
Dec 2009
Adirondack Mtns

3·372 Posts
Default How I Create a Colab Session That Extends Aliquot Sequences Working Directly with factordb

(Note: I expect to keep the first post of each of these "How I..." threads up-to-date with the latest version. Please read the rest of each thread to see what may have led to the current set of instructions.)

I will take the liberty of expecting readers to already be somewhat familiar with Google's Colaboratory sessions. There are several threads already on Colab and these should be reviewed by interested readers:

Google Colaboratory Notebook?
GPU72 Notebook Integration...
Notebook Instance Reverse SSH and HTTP Tunnels.
Colab question

I do not, as of yet, have a github account, so I have not created an upload of this to github. Others may feel free to do so, if desired.

The following is a manner to work with factordb.com to extend an Aliquot sequence. The session will retrieve the last composite in a sequence and factor it, returning the factors to factordb.com, which in turn extends the sequence. The session will ask for the initial sequence term to be entered. The session uses YAFU to perform the factoring, which in turn uses GMP-ECM for some of its process. The initial run installs all the necessary packages.

Note: Currently, this session is only using SIQS for factoring that GMP-ECM doesn't solve. The NFS switch is not used during compilation of YAFU and neither Msieve nor the GNFS sievers are installed.

To use Colab, you need a Gmail account and will be required to log into that account to run a session.

On to the specifics:

Open a Google Colaboratory session.
Sign in with your Google/Gmail account info.
Choose New notebook:
Code:
Menu->File->New notebook (or within popup)
Click Connect to start a session.
Edit title from Untitled... to whatever you like.
Paste the following into the Codeblock:
Code:
########################################################## 
### This Colaboratory session is designed to extend an ### 
### Aliquot sequence.  It works directly with factordb ### 
### and uses YAFU to find factors, which are then sent ### 
### back to factordb.                                  ### 
###                                                    ### 
### The only interaction with this session is to enter ### 
### the initial Aliquot term when prompted.            ### 
########################################################## 
  
import os 
import subprocess 
import urllib.request 
  
#reports factors to factordb 
def send2db(composite, factors): 
  factorline = str(factors) 
  sendline = 'report=' + str(composite) + '%3D' + factorline 
  dbcall = sendline.encode('utf-8') 
  temp2 = urllib.request.urlopen('http://factordb.com/report.php', dbcall) 
  
#checks to see if yafu already exists 
#if it does, this portion is skipped 
exists = os.path.isfile('yafu') 
if exists < 1: 
  print("Installing system packages. . .") 
#  subprocess.call(["chmod", "777", "/tmp"]) 
  subprocess.call(["apt", "update"]) 
  subprocess.call(["apt", "install", "g++", "m4", "make", "subversion", "libgmp-dev", "libtool", "p7zip", "autoconf"]) 
#retrieves ecm 
  print("Retrieving GMP-ECM. . .") 
  subprocess.call(["svn", "co", "svn://scm.gforge.inria.fr/svn/ecm/trunk", "ecm"]) 
  os.chdir("/content/ecm") 
  subprocess.call(["libtoolize"]) 
  subprocess.call(["autoreconf", "-i"]) 
  subprocess.call(["./configure", "--with-gmp=/usr/local/"]) 
  print("Compiling GMP-ECM. . .") 
  subprocess.call(["make"]) 
  subprocess.call(["make", "install"]) 
  print("Finished installing GMP-ECM. . .") 
  os.chdir("/content") 
#retrieves YAFU 
  print("Retrieving YAFU. . .") 
  subprocess.call(["svn", "co", "https://svn.code.sf.net/p/yafu/code/trunk", "/content/yafu"]) 
  os.chdir("/content/yafu") 
  subprocess.call(["mv", "yafu.ini", "yafu.ini.orig"]) 
  with open("yafu.ini", "a+") as yafuini: 
    yafuini.write("B1pm1=100000\n") 
    yafuini.write("B1pp1=20000\n") 
    yafuini.write("B1ecm=11000\n") 
    yafuini.write("rhomax=200\n") 
    yafuini.write("threads=2\n") 
    yafuini.write("pretest_ratio=0.25\n") 
    yafuini.write("ecm_path=/usr/local/bin/ecm\n") 
    yafuini.write("xover=93\n") 
    yafuini.write("no_clk_test=1") 
  print("Compiling YAFU. . .") 
  subprocess.call(["make", "x86_64", "USE_SSE41=1", "USE_AVX512=1"]) 
  print("Finished compiling YAFU. . .") 
  with open("/content/yafu/AliWork.sh", "a+") as AliWork: 
    AliWork.write("#!/bin/bash/\n") 
    AliWork.write("\n") 
    AliWork.write("function getComp(){\n") 
    AliWork.write("    wget -q -U Mozilla/5.0 \"http://factordb.com/sequences.php?se=1&aq=${seq}&action=last\" -O dbTemp\n") 
    AliWork.write("    exec <\"dbTemp\"\n") 
    AliWork.write("    while read line2\n") 
    AliWork.write("      do\n") 
    AliWork.write("        case $line2 in\n") 
    AliWork.write("          *\"index.php?id=\"*) ind=${line2##*002099}\n") 
    AliWork.write("          ind1=${#line2}\n") 
    AliWork.write("          let ind1=${ind1}-${#ind}\n") 
    AliWork.write("          let ind1=${ind1}-41\n") 
    AliWork.write("          ind2=http://factordb.com/index.php?id=${line2:${ind1}:19}\n") 
    AliWork.write("          ;;\n") 
    AliWork.write("        esac\n") 
    AliWork.write("      done\n") 
    AliWork.write("\n") 
    AliWork.write("    wget -q -U Mozilla/5.0 $ind2 -O dbTemp\n") 
    AliWork.write("\n") 
    AliWork.write("    exec <\"dbTemp\"\n") 
    AliWork.write("    while read line3\n") 
    AliWork.write("      do\n") 
    AliWork.write("        case $line3 in\n") 
    AliWork.write("          *\"query\"*) comp=${line3##*\'value=\"\'}\n") 
    AliWork.write("          comp2=${comp:0:${#comp}-2}\n") 
    AliWork.write("          complen=${#comp2}\n") 
    AliWork.write("          ;;\n") 
    AliWork.write("        esac\n") 
    AliWork.write("      done\n") 
    AliWork.write("}\n") 
    AliWork.write("\n") 
    AliWork.write("rm /content/yafu/stopAliWork 2>/dev/null\n") 
    AliWork.write("cd /content/yafu\n") 
    AliWork.write("test=\"\"\n") 
    AliWork.write("\n") 
    AliWork.write("if [ ${#1} -lt 3 ]\n") 
    AliWork.write("  then\n") 
    AliWork.write("    printf \"Enter seq#: \"\n") 
    AliWork.write("    read line in\n") 
    AliWork.write("    seq=$line\n") 
    AliWork.write("  else\n") 
    AliWork.write("    seq=$1\n") 
    AliWork.write("fi\n") 
    AliWork.write("\n") 
    AliWork.write("while [ ! -e ~/stopAliWork ]\n") 
    AliWork.write("  do\n") 
    AliWork.write("    rm *.log 2>/dev/null\n") 
    AliWork.write("    rm *.dat 2>/dev/null\n") 
    AliWork.write("    rm factors 2>/dev/null\n") 
    AliWork.write("    rm db* 2>/dev/null\n") 
    AliWork.write("    rm nfs.* 2>/dev/null\n") 
    AliWork.write("    getComp\n") 
    AliWork.write("\n")     
    AliWork.write("    echo sequence is $seq and composite is ${comp2:0:10}...${comp2:${complen}-2}\'<    \'${complen}\'>\'\n") 
    AliWork.write("    echo \"$comp2\" | ./yafu -of factors\n") 
    AliWork.write("    exec <\"factors\"\n") 
    AliWork.write("      read line in\n") 
    AliWork.write("    report=http://factordb.com/report.php?report=$(echo ${line:1:$complen}%3D${line:$complen+3} | tr -s \'/\' \'*\')\n") 
    AliWork.write("    wget -q -U Mozilla/5.0 \"$report\" -O dbRsave\n") 
    AliWork.write("\n") 
    AliWork.write("  done\n") 
print("All compilations completed!\n") 
%cd /content/yafu/ 
print("Enter sequence into box and press Enter:\n") 
!bash AliWork.sh
!rm ~/stopAliWork
Click on the Run cell icon or use CTRL-Enter. The first run will install packages for a few minutes and then prompt for the seq# (initial term). It will then work toward extending the sequence.

Subsequent runs should proceed without the compilation and installation steps.

Last fiddled with by EdH on 2021-02-12 at 19:27
EdH is offline   Reply With Quote