20220924, 22:33  #1  
"Ed Hall"
Dec 2009
Adirondack Mtns
11765_{8} Posts 
Work on a BASH Scipt to Find Polynomials Using the Best of Msieve and CADONFS
Max0526 helped me build a script for his polynomial spinning technique, but I was not able to fully realize all he did when spinning polynomials. My version of that script has never been effective with polynomials found by Gimarel who has consistently posted record scoring polynomials.
This thread is intended to use Gimarel's process, which he has been so kind as to share with us in the Polynomial Request thread, in building a script to implement his process. I have been stumped at the first step and hope to find a way through his process of using both Msieve and CADONFS to find the best polynomials we can, using the best portions from both packages. If possible, I would like to incorporate the use of a GPU in the first stage eventually, but I'm confused from the start with the following from the Polynomial Request thread: Quote:
At the start, I'm already confused by the above. If I run Msieve with np1, it will find the initial values and place them into msieve.dat.m. This file is not readable by CADONFS sopt (unless I'm missing some sopt switch(es)). My question at this point is whether to perform Msieve nps/npr to get to the polynomial format that sopt would use, or perhaps start with CADONFS (which would rule out a GPU) first? In trying to keep with one step at a time, I'd like to start with setting up the initial directory structure and work from there. My first step is to create a working directory at the same level as the Msieve and CADONFS directories and build the script to work within that directory: mkdir polyWork. In my case, the parent directory for the other three is Math and the following script will use that path. And, now for the script: Code:
#!/bin/bash ##################################### ## This script is designed to use ## ## portions of Msieve and CADONFS ## ## to find polynomials. ## ##################################### # set path to polyWork directory: cd $HOME/Math/polyWork # Set values for Msieve and CADONFS # operations: minc=1000000 maxc=1200000 if [ ${#1} gt 9 ] then composite=$1 elif [ e worktodo.ini ] then read composite <worktodo.ini else echo n "Enter composite: " read composite in fi echo "$composite" >worktodo.ini echo "Finding polynomials for: $composite" # Since this location should be relative # to the working directory and static, it # is only set in the command lines. ../msieve/msieve v t 20 np1 "${minc}, ${maxc}" echo "Msieve stage 1 complete!" 1. The min and max coeff range is not observed when I run only np1 2. The resulting msieve.dat.m file is not readable by CADONFS sopt, unless there's an option I'm unaware of. Last fiddled with by EdH on 20220924 at 23:08 

20220924, 22:48  #2 
Sep 2008
Kansas
3,761 Posts 

20220924, 23:09  #3 
"Ed Hall"
Dec 2009
Adirondack Mtns
1001111110101_{2} Posts 

20220925, 06:30  #4  
Apr 2010
2^{2}×61 Posts 
Quote:
1. Run msieve nps with a really high stage2_norm and convert the output with something like that for degree six: Code:
cat msieve.dat.ms  while read l; do line=($l) echo "n: $composite" echo "c6: ${line[0]}" echo "c5: ${line[1]}" echo "c4: ${line[2]}" echo "c3: ${line[3]}" echo "c2: ${line[4]}" echo "c1: ${line[5]}" echo "c0: ${line[6]}" echo "Y1: ${line[7]}" echo "Y0: ${line[8]}" echo done Code:
Index: gnfs/poly/stage2/stage2.c ===================================================================  gnfs/poly/stage2/stage2.c (revision 1035) +++ gnfs/poly/stage2/stage2.c (working copy) @@ 219,10 +219,12 @@ void poly_sizeopt_run(poly_sizeopt_t *data, mpz_t ad, mpz_t p, mpz_t d) { + int32 i; double pol_norm; double alpha_proj; int status; curr_poly_t *c = (curr_poly_t *)(data>internal); + FILE *mfile = (FILE *)data>callback_data; mpz_set(c>gmp_d, d); mpz_set(c>gmp_p, p); @@ 237,6 +239,13 @@ return; } + gmp_fprintf(mfile, "n: %Zd\n", data>gmp_N); + for (i = data>degree; i >= 0; i) + gmp_fprintf(mfile, "c%d: %Zd\n", i, c>gmp_a[i]); + gmp_fprintf(mfile, "Y1: %Zd\nY0: %Zd\n\n", c>gmp_lina[1], c>gmp_lina[0]); + fflush(mfile); + return; + optimize_initial(c, data>degree, &pol_norm, 0); stage2_root_score(data>degree, c>gmp_a, 100, &alpha_proj, 1); 

20220925, 12:26  #5 
"Ed Hall"
Dec 2009
Adirondack Mtns
3×13×131 Posts 
Thanks! The conversion wasn't my block. I commonly convert* for versions of my spin script. There are also some programs for that. My confusion was how to get to that point without running Msieve size option. Now I understand that I do run the size option with a high enough stage2_norm to only save the top scoring polynomials for the CADONFS sopt run.
I'll work some more and add another step to see where I get. There is some mention elsewhere about adjusting the stage1 and stage2 norms from Msieve's original calculations. I'll add that as well. And, then you can let me know if I'm getting way off course. Of note: To get the GPU portion to work, I'm going to move everything from the "polyWork" directory into an Msieve directory called "msievePoly" * My normal conversion accepts all coefficients up through degree 8, just in case. 
20220925, 12:59  #6  
Apr 2010
2^{2}·61 Posts 
Quote:


20220925, 13:05  #7  
"Ed Hall"
Dec 2009
Adirondack Mtns
13F5_{16} Posts 
Quote:
Would there be a suggestion for the stage1 and stage2 starting values, either static or based on the original Msieve ones? 

20220926, 05:23  #8  
Apr 2010
2^{2}×61 Posts 
Quote:
As I don't use msieve's sizeopt any more, I don't have a suggestion for stage 2 as well. If you just want all hits use something like 1e40. 

20220926, 12:14  #9 
"Ed Hall"
Dec 2009
Adirondack Mtns
3×13×131 Posts 
Thanks! I've gotten quite a ways past the single next step.
Here's where I am ATM:  I run Msieve np1 for 5 seconds to get initial values for degree and stage2_norm.  I run Msieve with np1 and nps using a stage2_norm based on a multiplier times the original.   This seems to only run single threaded. Should I be able to get multithreaded?  I convert the msieve.dat.ms file into a poly file for CADONFS sopt.   I've made this portion work across varying degrees by incorporating the degree value found earlier.   This file is tens for thousands of polynomials.    Should I use the scores to trim this yet?  I run CADONFS sopt on the new file.   You mention using exp_E to determine value.    IIRC, the lower exp_E, the better (or am I backwards again?).     I'm thinking at this point I should cull somewhat.     Would it be of value to only keep the polynomials with the lowest exp_E?   Should I make more than one pass with varying values for sopt or is there a particular value I should use? 
20220926, 15:30  #11 
"Curtis"
Feb 2005
Riverside, CA
2^{2}·3·463 Posts 
If I were going this route, I think I'd want to feed CADO sopt thousands of hits but not tens of thousands. The stage1 norm is still a useful filter to keep only the best GPU stage 1 hits.
The advice I gave in that linked post was intended to feed the CPU a stream of GPU hits such that sizeopt on CPU took about the same time as stage 1 on GPU. You might want more hits than that, but I'd still reduce the default stage1 norm by a factor of 5 to start, and then tune as desired to balance the GPU time spent vs the CADOsopt time spent on CPU. I personally lean toward making the GPU work "harder" by setting the stage1 norm lower. 
Thread Tools  
Similar Threads  
Thread  Thread Starter  Forum  Replies  Last Post 
Calculator that can factor and find exact roots of polynomials  alpertron  Programming  39  20220602 12:21 
Improving Polynomials With CADONFS and Msieve Tools  EdH  Factoring  4  20211018 14:29 
Combining Msieve with CADO NFS  mfeltz  Msieve  10  20160316 21:12 
How to find values of polynomials with nice factorization?  Drdmitry  Computer Science & Computational Number Theory  18  20150910 12:23 
how to run msieve or cadonfs on mpicluster?  ravlyuchenko  Msieve  1  20110816 12:12 