mersenneforum.org > YAFU YAFU 2.0
 Register FAQ Search Today's Posts Mark Forums Read

2021-09-16, 17:10   #243
bsquared

"Ben"
Feb 2007

2·11·163 Posts

Quote:
 Originally Posted by EdH Just an additional note that the other Xeon acted the same. I forgot to include it in the last post.
Try the most-recent commit (just now). It is working for me...

2021-09-16, 17:20   #244
bsquared

"Ben"
Feb 2007

2·11·163 Posts

Quote:
 Originally Posted by James Heinrich I'll admit I'm posting this while looking at v1.34.5 and not v2.07, but is it possible to add in some kind of rough ETA to SIQS, which currently displays something like:
Not trivial, given the ability to restart jobs and the non-linear growth of relation-sets in DLP/3LP variations.

Very roughly, when the number of fulls from partials is equal to the number of fulls then you are about 2/3 to 3/4 done.

2021-09-16, 17:27   #245
James Heinrich

"James Heinrich"
May 2004
ex-Northern Ontario

70148 Posts

Quote:
 Originally Posted by bsquared Not trivial... then you are about 2/3 to 3/4 done.
I figured it's hard to get a precise number. But just wondering if something "very approximate" could be contrived, something like "about 3-6 hours left" or something like that?

2021-09-16, 22:18   #246
EdH

"Ed Hall"
Dec 2009

47×89 Posts

Quote:
 Originally Posted by bsquared Try the most-recent commit (just now). It is working for me...
Probably due to my CPUs missing (almost) all the valuable flags, I got an error after "make clean NFS=clean USE_SSE41=clean":
Code:
$make NFS=1 USE_SSE41=1 . . . factor/avx-ecm/avx_ecm.h:836:34: error: unknown type name \u2018__m512i\u2019 836 | void vecmul52_1(vec_bignum_t* a, __m512i b, vec_bignum_t* c, vec_bignum_t* n, vec_bignum_t* s, vec_monty_t* mdata); | ^~~~~~~ . . . make: *** [Makefile:393: factor/avx-ecm/avxecm.o] Error 1 I also tried without the sse4_1 option and received the same error message. On a different subject, I'm trying to figure out how to send options in the following script line: Code: echo "factor(${composite})" | ./yafu
I've tried adding -one and -silent before and after the pipe (at the ends of each command) and inside and outside the quotes, with and without another set of enclosing quotes. Can these be added to the script line, or can I place them in yafu.ini in some manner? I'm using separate copies of yafu and yafu.ini within the working directory where the script is located, so I'm free to modify yafu.ini to be specific. Alternately for -silent, is there a less verbose mode?

Sorry if I'm a pain. Thanks for all the help.

2021-09-16, 22:33   #247
James Heinrich

"James Heinrich"
May 2004
ex-Northern Ontario

22·29·31 Posts

Quote:
 Originally Posted by James Heinrich 1) ECM runs as a single thread of yafu-x64.exe
Just confirming that internal ECM now uses the appropriate number of threads in v2.07 (Win10), thanks.

I did notice something when checking that -- I'm specifying -p on the commandline but yafu-x64.exe is running in "normal" priority, not idle (p is also set in yafu.ini).
Attached Thumbnails

 2021-09-17, 12:30 #248 EdH     "Ed Hall" Dec 2009 Adirondack Mtns 47×89 Posts Revision 2.07 appears to be stuck in a loop: Code: . . . starting SIQS on c88: 1072980570200329351188730228988024470047893288038112575153917530948170108341537641314357 70449 relations found: 20728 full + 49721 from 872363 partial threw away 0 relations with large primes too small ==== sieve params ==== n = 88 digits, 290 bits factor base: 66560 primes (max prime = 1773997) single large prime cutoff: 195139670 (110 * pmax) double large prime range from 3147065356009 to 2173398172298490 DLP MFB = 1.85 using SSE41 enabled 32k sieve core sieve interval: 12 blocks of size 32768 polynomial A has ~ 11 factors using multiplier of 1 (kn mod 8 == 5) using SPV correction of 20 bits, starting at offset 35 trial factoring cutoff at 93 bits ==== sieving in progress ( 24 threads): 66624 relations needed ==== ==== Press ctrl-c to abort and save state ==== 70449 rels found: 20728 full + 49721 from 872363 partial, (1592093.11 rels/sec) sieving required 0 total polynomials (4294967295 'A' polynomials) trial division touched 0 sieve locations out of 0 dlp-ecm: 0 failures, 0 attempts, 0 outside range, 0 prp, 0 useful total reports = 0, total surviving reports = 0 total blocks sieved = 0, avg surviving reports per block = -nan Elapsed time: 0.5610 sec QS elapsed time = 0.5611 seconds. ==== post processing stage (msieve-1.38) ==== read 893090 relations begin singleton removal with 893090 relations reduce to 162769 relations in 9 passes attempting to read and process 162769 relations recovered 162769 relations recovered 129773 polynomials attempting to build 70448 cycles found 70448 cycles from 162769 relations in 5 passes distribution of cycle lengths: length 1 : 20728 length 2 : 15380 length 3 : 12812 length 4 : 8869 length 5 : 5644 length 6 : 3303 length 7 : 1814 length 9+: 1898 largest cycle: 18 relations matrix is 66560 x 70448 (16.6 MB) with weight 3794245 (53.86/col) sparse part has weight 3794245 (53.86/col) filtering completed in 4 passes matrix is 59263 x 59327 (13.3 MB) with weight 3021409 (50.93/col) sparse part has weight 3021409 (50.93/col) saving the first 48 matrix rows for later matrix is 59215 x 59327 (11.6 MB) with weight 2672491 (45.05/col) sparse part has weight 2454774 (41.38/col) matrix includes 64 packed rows using block size 23730 for processor cache size 15360 kB commencing Lanczos iteration memory use: 9.5 MB lanczos halted after 938 iterations (dim = 59210) recovered 14 nontrivial dependencies Lanczos elapsed time = 14.3076 seconds. Sqrt elapsed time = 1.3032 seconds. SIQS elapsed time = 16.1745 seconds. pretesting / qs ratio was 0.90 fac: check tune params contained invalid parameter(s), ignoring tune info. fac: setting target pretesting digits to 27.08 t15: 67.74 t20: 29.98 t25: 4.05 t30: 0.42 t35: 0.03 fac: estimated sum of completed work is t27.08 starting SIQS on c88: 1072980570200329351188730228988024470047893288038112575153917530948170108341537641314357 70449 relations found: 20728 full + 49721 from 872363 partial threw away 0 relations with large primes too small ==== sieve params ==== n = 88 digits, 290 bits factor base: 66560 primes (max prime = 1773997) single large prime cutoff: 195139670 (110 * pmax) double large prime range from 3147065356009 to 2173398172298490 DLP MFB = 1.85 using SSE41 enabled 32k sieve core sieve interval: 12 blocks of size 32768 polynomial A has ~ 11 factors using multiplier of 1 (kn mod 8 == 5) using SPV correction of 20 bits, starting at offset 35 trial factoring cutoff at 93 bits ==== sieving in progress ( 24 threads): 66624 relations needed ==== ==== Press ctrl-c to abort and save state ==== 70449 rels found: 20728 full + 49721 from 872363 partial, (1348970.02 rels/sec) sieving required 0 total polynomials (4294967295 'A' polynomials) trial division touched 0 sieve locations out of 0 dlp-ecm: 0 failures, 0 attempts, 0 outside range, 0 prp, 0 useful total reports = 0, total surviving reports = 0 total blocks sieved = 0, avg surviving reports per block = -nan Elapsed time: 0.6621 sec QS elapsed time = 0.6622 seconds. ==== post processing stage (msieve-1.38) ==== read 893090 relations begin singleton removal with 893090 relations reduce to 162769 relations in 9 passes attempting to read and process 162769 relations recovered 162769 relations recovered 129773 polynomials attempting to build 70448 cycles found 70448 cycles from 162769 relations in 5 passes distribution of cycle lengths: length 1 : 20728 length 2 : 15380 length 3 : 12812 length 4 : 8869 length 5 : 5644 length 6 : 3303 length 7 : 1814 length 9+: 1898 largest cycle: 18 relations matrix is 66560 x 70448 (16.6 MB) with weight 3794245 (53.86/col) sparse part has weight 3794245 (53.86/col) filtering completed in 4 passes matrix is 59263 x 59327 (13.3 MB) with weight 3021409 (50.93/col) sparse part has weight 3021409 (50.93/col) saving the first 48 matrix rows for later matrix is 59215 x 59327 (11.6 MB) with weight 2672491 (45.05/col) sparse part has weight 2454774 (41.38/col) matrix includes 64 packed rows using block size 23730 for processor cache size 15360 kB commencing Lanczos iteration memory use: 9.5 MB lanczos halted after 938 iterations (dim = 59210) recovered 14 nontrivial dependencies Lanczos elapsed time = 13.7053 seconds. Sqrt elapsed time = 1.0454 seconds. SIQS elapsed time = 15.4167 seconds. pretesting / qs ratio was 0.94 fac: check tune params contained invalid parameter(s), ignoring tune info. fac: setting target pretesting digits to 27.08 t15: 67.74 t20: 29.98 t25: 4.05 t30: 0.42 t35: 0.03 fac: estimated sum of completed work is t27.08 starting SIQS on c88: 1072980570200329351188730228988024470047893288038112575153917530948170108341537641314357 70449 relations found: 20728 full + 49721 from 872363 partial threw away 0 relations with large primes too small ==== sieve params ==== n = 88 digits, 290 bits factor base: 66560 primes (max prime = 1773997) single large prime cutoff: 195139670 (110 * pmax) double large prime range from 3147065356009 to 2173398172298490 DLP MFB = 1.85 using SSE41 enabled 32k sieve core sieve interval: 12 blocks of size 32768 polynomial A has ~ 11 factors using multiplier of 1 (kn mod 8 == 5) using SPV correction of 20 bits, starting at offset 35 trial factoring cutoff at 93 bits ==== sieving in progress ( 24 threads): 66624 relations needed ==== ==== Press ctrl-c to abort and save state ==== 70449 rels found: 20728 full + 49721 from 872363 partial, (1616530.37 rels/sec) sieving required 0 total polynomials (4294967295 'A' polynomials) trial division touched 0 sieve locations out of 0 dlp-ecm: 0 failures, 0 attempts, 0 outside range, 0 prp, 0 useful total reports = 0, total surviving reports = 0 total blocks sieved = 0, avg surviving reports per block = -nan Elapsed time: 0.5525 sec QS elapsed time = 0.5526 seconds. ==== post processing stage (msieve-1.38) ==== read 893090 relations begin singleton removal with 893090 relations reduce to 162769 relations in 9 passes attempting to read and process 162769 relations recovered 162769 relations recovered 129773 polynomials attempting to build 70448 cycles found 70448 cycles from 162769 relations in 5 passes distribution of cycle lengths: length 1 : 20728 length 2 : 15380 length 3 : 12812 length 4 : 8869 length 5 : 5644 length 6 : 3303 length 7 : 1814 length 9+: 1898 largest cycle: 18 relations matrix is 66560 x 70448 (16.6 MB) with weight 3794245 (53.86/col) sparse part has weight 3794245 (53.86/col) filtering completed in 4 passes matrix is 59263 x 59327 (13.3 MB) with weight 3021409 (50.93/col) sparse part has weight 3021409 (50.93/col) saving the first 48 matrix rows for later matrix is 59215 x 59327 (11.6 MB) with weight 2672491 (45.05/col) sparse part has weight 2454774 (41.38/col) matrix includes 64 packed rows using block size 23730 for processor cache size 15360 kB commencing Lanczos iteration memory use: 9.5 MB lanczos halted after 938 iterations (dim = 59210) recovered 14 nontrivial dependencies Lanczos elapsed time = 13.9592 seconds. Sqrt elapsed time = 1.0246 seconds. SIQS elapsed time = 15.5372 seconds. pretesting / qs ratio was 0.93 fac: check tune params contained invalid parameter(s), ignoring tune info. fac: setting target pretesting digits to 27.08 t15: 67.74 t20: 29.98 t25: 4.05 t30: 0.42 t35: 0.03 fac: estimated sum of completed work is t27.08 . . . Edit: This is occurring on 2 of 6 machines. Last fiddled with by EdH on 2021-09-17 at 12:36
2021-09-17, 15:50   #249
chris2be8

Sep 2009

1000101011012 Posts

Quote:
 Originally Posted by EdH On a different subject, I'm trying to figure out how to send options in the following script line: Code: echo "factor(${composite})" | ./yafu I've tried adding -one and -silent before and after the pipe (at the ends of each command) and inside and outside the quotes, with and without another set of enclosing quotes. Can these be added to the script line, or can I place them in yafu.ini in some manner? I'm using separate copies of yafu and yafu.ini within the working directory where the script is located, so I'm free to modify yafu.ini to be specific. Alternately for -silent, is there a less verbose mode? My perl scripts use: Code: $cmd="\"$YAFU\" 'factor($_[0])' -p -ecm_path /usr/bin/ecm -logfile $LOGFILE -threads$NUM_CPUS";
...
$res=system($cmd);
perl substitutes suitable values for $YAFU,$_[0], $LOGFILE and$NUM_CPUS, then passes the resulting string to the shell.

Or a trivial example from the command line:
Code:
chris@rigel:~/bin$yafu 'factor(2^128+1)' -threads 2 fac: factoring 340282366920938463463374607431768211457 fac: using pretesting plan: normal fac: using tune info for qs/gnfs crossover div: primes less than 10000 rho: x^2 + 3, starting 1000 iterations on C39 rho: x^2 + 2, starting 1000 iterations on C39 rho: x^2 + 1, starting 1000 iterations on C39 pm1: starting B1 = 150K, B2 = gmp-ecm default on C39 ecm: 30/30 curves on C39, B1=2K, B2=gmp-ecm default starting SIQS on c39: 340282366920938463463374607431768211457 ==== sieving in progress ( 2 threads): 656 relations needed ==== ==== Press ctrl-c to abort and save state ==== 539 rels found: 297 full + 242 from 2231 partial, (34049.89 rels/sec) SIQS elapsed time = 0.1017 seconds. Total factoring time = 0.4423 seconds ***factors found*** P22 = 5704689200685129054721 P17 = 59649589127497217 ans = 1 Note it used 2 threads for sieving, so it saw the parameter -threads 2 2021-09-17, 18:48 #250 bsquared "Ben" Feb 2007 2×11×163 Posts Quote:  Originally Posted by EdH Probably due to my CPUs missing (almost) all the valuable flags, I got an error after "make clean NFS=clean USE_SSE41=clean":[code]$ make NFS=1 USE_SSE41=1 . . . factor/avx-ecm/avx_ecm.h:836:34: error: unknown type name \u2018__m512i\u2019 836 | void vecmul52_1(vec_bignum_t* a, __m512i b, vec_bignum_t* c, vec_bignum_t* n, vec_bignum_t* s, vec_monty_t* mdata); | ^~~~~~~ . . . Sorry if I'm a pain. Thanks for all the help.
No worries at all. You just caught me in the middle of other intermediate updates, where I don't even do my usual poor job of checking if things work on other architectures. Try the latest commit now.

Quote:
 Originally Posted by EdH Revision 2.07 appears to be stuck in a loop:
You got me with this one, no idea what's going on. Maybe some context? Is this a resume or did it run through from scratch? Running on cmd line or in interpreter? etc.

2021-09-17, 21:01   #251
EdH

"Ed Hall"
Dec 2009

47·89 Posts

Quote:
 Originally Posted by chris2be8 My perl scripts use: Code: $cmd="\"$YAFU\" 'factor($_[0])' -p -ecm_path /usr/bin/ecm -logfile$LOGFILE -threads $NUM_CPUS"; ...$res=system($cmd); perl substitutes suitable values for$YAFU, $_[0],$LOGFILE and \$NUM_CPUS, then passes the resulting string to the shell. . . .
Thanks. I'll try some more syntax variations with my script to see if I can match your perl call.

Quote:
 Originally Posted by bsquared No worries at all. You just caught me in the middle of other intermediate updates, where I don't even do my usual poor job of checking if things work on other architectures. Try the latest commit now. You got me with this one, no idea what's going on. Maybe some context? Is this a resume or did it run through from scratch? Running on cmd line or in interpreter? etc.
Excellent! Thanks! All seems well.

Note: I found that tiny, unobtrusive, miniature, lower case, "v" you had hidden in the yafu.ini file. So the verbosity is back to my liking.

As for the looping machines, they were running a bash script that feeds YAFU a composite. Both had already completed a good number of composites via the script. The one machine I noted had been looping for almost 7 hours overnight. I just fed that composite to the new version as a test and it flew through it. (Of course, I expected it would.)

Thanks for all!

 2021-09-18, 00:08 #252 EdH     "Ed Hall" Dec 2009 Adirondack Mtns 47·89 Posts Still Have Looping I seemed to still have looping on one of the same machines as before, but I did break out of it without CTRL-c: Code: . . . starting SIQS on c89: 13552335418571691348826103661651040263240362626711372932187961446062171132450991836093689 ==== sieving in progress ( 24 threads): 68592 relations needed ==== ==== Press ctrl-c to abort and save state ==== 71951 rels found: 21038 full + 50913 from 925085 partial, (1651661.66 rels/sec) SIQS elapsed time = 17.1809 seconds. starting SIQS on c89: 13552335418571691348826103661651040263240362626711372932187961446062171132450991836093689 ==== sieving in progress ( 24 threads): 68592 relations needed ==== ==== Press ctrl-c to abort and save state ==== 71951 rels found: 21038 full + 50913 from 925085 partial, (1610592.10 rels/sec) SIQS elapsed time = 16.3661 seconds. starting SIQS on c89: 13552335418571691348826103661651040263240362626711372932187961446062171132450991836093689 ==== sieving in progress ( 24 threads): 68592 relations needed ==== ==== Press ctrl-c to abort and save state ==== 71951 rels found: 21038 full + 50913 from 925085 partial, (1650011.16 rels/sec) SIQS elapsed time = 18.6747 seconds. . . . I noticed the rels found was already populated, so I used CTRL-z to pause YAFU and deleted siqs.dat. Then I resumed with "fg" and the SIQS operation started from zero rels and completed the factoring. I've added "rm siqs.dat" to my script so there shouldn't be any leftover relations from previous runs. I'll see if this machine loops anymore.
 2021-09-18, 18:39 #253 EdH     "Ed Hall" Dec 2009 Adirondack Mtns 47·89 Posts An exciting day! I just updated all my machines, Core2 , i3, i5, i7, AMD (without SSE4_1) and Xeons. (I had to make a script.) All are now running YAFU 2.07!! A couple questions: - I didn't, but should I rebuild the Core2 machines with the Core2 line in the Makefile uncommented? - In ytools, should the "CC = gcc-7.3.0" be changed to match the "CC = gcc" in the ysieve and YAFU Makefiles? Thanks for all!

 Similar Threads Thread Thread Starter Forum Replies Last Post chris2be8 YAFU 6 2019-10-17 16:22 EdH YAFU 8 2018-03-14 17:22 bsquared YAFU 119 2015-11-05 16:24 storflyt32 YAFU 2 2015-06-29 05:19 bsquared YAFU 28 2012-07-20 16:17

All times are UTC. The time now is 10:57.

Wed Jan 19 10:57:07 UTC 2022 up 180 days, 5:26, 0 users, load averages: 1.74, 1.64, 1.51