YAFU 2.0 - Page 23

bsquared · 2021-09-16, 17:10

Quote:

Originally Posted by EdH

Just an additional note that the other Xeon acted the same. I forgot to include it in the last post.

Try the most-recent commit (just now). It is working for me...

bsquared · 2021-09-16, 17:20

Quote:

Originally Posted by James Heinrich

I'll admit I'm posting this while looking at v1.34.5 and not v2.07, but is it possible to add in some kind of rough ETA to SIQS, which currently displays something like:

Not trivial, given the ability to restart jobs and the non-linear growth of relation-sets in DLP/3LP variations.

Very roughly, when the number of fulls from partials is equal to the number of fulls then you are about 2/3 to 3/4 done.

James Heinrich · 2021-09-16, 17:27

Quote:

Originally Posted by bsquared

Not trivial... then you are about 2/3 to 3/4 done.

I figured it's hard to get a precise number. But just wondering if something "very approximate" could be contrived, something like "about 3-6 hours left" or something like that?

EdH · 2021-09-16, 22:18

Quote:

Originally Posted by bsquared

Try the most-recent commit (just now). It is working for me...

Probably due to my CPUs missing (almost) all the valuable flags, I got an error after "make clean NFS=clean USE_SSE41=clean":

Code:

$ make NFS=1 USE_SSE41=1
. . .
factor/avx-ecm/avx_ecm.h:836:34: error: unknown type name \u2018__m512i\u2019
  836 | void vecmul52_1(vec_bignum_t* a, __m512i b, vec_bignum_t* c, vec_bignum_t* n, vec_bignum_t* s, vec_monty_t* mdata);
      |                                  ^~~~~~~
. . .
make: *** [Makefile:393: factor/avx-ecm/avxecm.o] Error 1

I also tried without the sse4_1 option and received the same error message.

On a different subject, I'm trying to figure out how to send options in the following script line:

Code:

echo "factor(${composite})" | ./yafu

I've tried adding -one and -silent before and after the pipe (at the ends of each command) and inside and outside the quotes, with and without another set of enclosing quotes. Can these be added to the script line, or can I place them in yafu.ini in some manner? I'm using separate copies of yafu and yafu.ini within the working directory where the script is located, so I'm free to modify yafu.ini to be specific. Alternately for -silent, is there a less verbose mode?

Sorry if I'm a pain. Thanks for all the help.

James Heinrich · 2021-09-16, 22:33

Quote:

Originally Posted by James Heinrich

1) ECM runs as a single thread of yafu-x64.exe

Just confirming that internal ECM now uses the appropriate number of threads in v2.07 (Win10), thanks.

I did notice something when checking that -- I'm specifying -p on the commandline but yafu-x64.exe is running in "normal" priority, not idle (p is also set in yafu.ini).

EdH · 2021-09-17, 12:30

Revision 2.07 appears to be stuck in a loop:

Code:

. . .
starting SIQS on c88: 1072980570200329351188730228988024470047893288038112575153917530948170108341537641314357
70449 relations found: 20728 full + 49721 from 872363 partial
threw away 0 relations with large primes too small

==== sieve params ====
n = 88 digits, 290 bits
factor base: 66560 primes (max prime = 1773997)
single large prime cutoff: 195139670 (110 * pmax)
double large prime range from 3147065356009 to 2173398172298490
DLP MFB = 1.85
using SSE41 enabled 32k sieve core
sieve interval: 12 blocks of size 32768
polynomial A has ~ 11 factors
using multiplier of 1 (kn mod 8 == 5)
using SPV correction of 20 bits, starting at offset 35
trial factoring cutoff at 93 bits

==== sieving in progress ( 24 threads):   66624 relations needed ====
====             Press ctrl-c to abort and save state            ====
70449 rels found: 20728 full + 49721 from 872363 partial, (1592093.11 rels/sec)

sieving required 0 total polynomials (4294967295 'A' polynomials)
trial division touched 0 sieve locations out of 0
dlp-ecm: 0 failures, 0 attempts, 0 outside range, 0 prp, 0 useful
total reports = 0, total surviving reports = 0
total blocks sieved = 0, avg surviving reports per block = -nan
Elapsed time: 0.5610 sec
QS elapsed time = 0.5611 seconds.

==== post processing stage (msieve-1.38) ====
read 893090 relations
begin singleton removal with 893090 relations
reduce to 162769 relations in 9 passes
attempting to read and process 162769 relations
recovered 162769 relations
recovered 129773 polynomials
attempting to build 70448 cycles
found 70448 cycles from 162769 relations in 5 passes
distribution of cycle lengths:
   length 1 : 20728
   length 2 : 15380
   length 3 : 12812
   length 4 : 8869
   length 5 : 5644
   length 6 : 3303
   length 7 : 1814
   length 9+: 1898
largest cycle: 18 relations
matrix is 66560 x 70448 (16.6 MB) with weight 3794245 (53.86/col)
sparse part has weight 3794245 (53.86/col)
filtering completed in 4 passes
matrix is 59263 x 59327 (13.3 MB) with weight 3021409 (50.93/col)
sparse part has weight 3021409 (50.93/col)
saving the first 48 matrix rows for later
matrix is 59215 x 59327 (11.6 MB) with weight 2672491 (45.05/col)
sparse part has weight 2454774 (41.38/col)
matrix includes 64 packed rows
using block size 23730 for processor cache size 15360 kB
commencing Lanczos iteration
memory use: 9.5 MB
lanczos halted after 938 iterations (dim = 59210)
recovered 14 nontrivial dependencies
Lanczos elapsed time = 14.3076 seconds.
Sqrt elapsed time = 1.3032 seconds.
SIQS elapsed time = 16.1745 seconds.
pretesting / qs ratio was 0.90
fac: check tune params contained invalid parameter(s), ignoring tune info.
fac: setting target pretesting digits to 27.08
    t15: 67.74
    t20: 29.98
    t25: 4.05
    t30: 0.42
    t35: 0.03
fac: estimated sum of completed work is t27.08

starting SIQS on c88: 1072980570200329351188730228988024470047893288038112575153917530948170108341537641314357
70449 relations found: 20728 full + 49721 from 872363 partial
threw away 0 relations with large primes too small

==== sieve params ====
n = 88 digits, 290 bits
factor base: 66560 primes (max prime = 1773997)
single large prime cutoff: 195139670 (110 * pmax)
double large prime range from 3147065356009 to 2173398172298490
DLP MFB = 1.85
using SSE41 enabled 32k sieve core
sieve interval: 12 blocks of size 32768
polynomial A has ~ 11 factors
using multiplier of 1 (kn mod 8 == 5)
using SPV correction of 20 bits, starting at offset 35
trial factoring cutoff at 93 bits

==== sieving in progress ( 24 threads):   66624 relations needed ====
====             Press ctrl-c to abort and save state            ====
70449 rels found: 20728 full + 49721 from 872363 partial, (1348970.02 rels/sec)

sieving required 0 total polynomials (4294967295 'A' polynomials)
trial division touched 0 sieve locations out of 0
dlp-ecm: 0 failures, 0 attempts, 0 outside range, 0 prp, 0 useful
total reports = 0, total surviving reports = 0
total blocks sieved = 0, avg surviving reports per block = -nan
Elapsed time: 0.6621 sec
QS elapsed time = 0.6622 seconds.

==== post processing stage (msieve-1.38) ====
read 893090 relations
begin singleton removal with 893090 relations
reduce to 162769 relations in 9 passes
attempting to read and process 162769 relations
recovered 162769 relations
recovered 129773 polynomials
attempting to build 70448 cycles
found 70448 cycles from 162769 relations in 5 passes
distribution of cycle lengths:
   length 1 : 20728
   length 2 : 15380
   length 3 : 12812
   length 4 : 8869
   length 5 : 5644
   length 6 : 3303
   length 7 : 1814
   length 9+: 1898
largest cycle: 18 relations
matrix is 66560 x 70448 (16.6 MB) with weight 3794245 (53.86/col)
sparse part has weight 3794245 (53.86/col)
filtering completed in 4 passes
matrix is 59263 x 59327 (13.3 MB) with weight 3021409 (50.93/col)
sparse part has weight 3021409 (50.93/col)
saving the first 48 matrix rows for later
matrix is 59215 x 59327 (11.6 MB) with weight 2672491 (45.05/col)
sparse part has weight 2454774 (41.38/col)
matrix includes 64 packed rows
using block size 23730 for processor cache size 15360 kB
commencing Lanczos iteration
memory use: 9.5 MB
lanczos halted after 938 iterations (dim = 59210)
recovered 14 nontrivial dependencies
Lanczos elapsed time = 13.7053 seconds.
Sqrt elapsed time = 1.0454 seconds.
SIQS elapsed time = 15.4167 seconds.
pretesting / qs ratio was 0.94
fac: check tune params contained invalid parameter(s), ignoring tune info.
fac: setting target pretesting digits to 27.08
    t15: 67.74
    t20: 29.98
    t25: 4.05
    t30: 0.42
    t35: 0.03
fac: estimated sum of completed work is t27.08

starting SIQS on c88: 1072980570200329351188730228988024470047893288038112575153917530948170108341537641314357
70449 relations found: 20728 full + 49721 from 872363 partial
threw away 0 relations with large primes too small

==== sieve params ====
n = 88 digits, 290 bits
factor base: 66560 primes (max prime = 1773997)
single large prime cutoff: 195139670 (110 * pmax)
double large prime range from 3147065356009 to 2173398172298490
DLP MFB = 1.85
using SSE41 enabled 32k sieve core
sieve interval: 12 blocks of size 32768
polynomial A has ~ 11 factors
using multiplier of 1 (kn mod 8 == 5)
using SPV correction of 20 bits, starting at offset 35
trial factoring cutoff at 93 bits

==== sieving in progress ( 24 threads):   66624 relations needed ====
====             Press ctrl-c to abort and save state            ====
70449 rels found: 20728 full + 49721 from 872363 partial, (1616530.37 rels/sec)

sieving required 0 total polynomials (4294967295 'A' polynomials)
trial division touched 0 sieve locations out of 0
dlp-ecm: 0 failures, 0 attempts, 0 outside range, 0 prp, 0 useful
total reports = 0, total surviving reports = 0
total blocks sieved = 0, avg surviving reports per block = -nan
Elapsed time: 0.5525 sec
QS elapsed time = 0.5526 seconds.

==== post processing stage (msieve-1.38) ====
read 893090 relations
begin singleton removal with 893090 relations
reduce to 162769 relations in 9 passes
attempting to read and process 162769 relations
recovered 162769 relations
recovered 129773 polynomials
attempting to build 70448 cycles
found 70448 cycles from 162769 relations in 5 passes
distribution of cycle lengths:
   length 1 : 20728
   length 2 : 15380
   length 3 : 12812
   length 4 : 8869
   length 5 : 5644
   length 6 : 3303
   length 7 : 1814
   length 9+: 1898
largest cycle: 18 relations
matrix is 66560 x 70448 (16.6 MB) with weight 3794245 (53.86/col)
sparse part has weight 3794245 (53.86/col)
filtering completed in 4 passes
matrix is 59263 x 59327 (13.3 MB) with weight 3021409 (50.93/col)
sparse part has weight 3021409 (50.93/col)
saving the first 48 matrix rows for later
matrix is 59215 x 59327 (11.6 MB) with weight 2672491 (45.05/col)
sparse part has weight 2454774 (41.38/col)
matrix includes 64 packed rows
using block size 23730 for processor cache size 15360 kB
commencing Lanczos iteration
memory use: 9.5 MB
lanczos halted after 938 iterations (dim = 59210)
recovered 14 nontrivial dependencies
Lanczos elapsed time = 13.9592 seconds.
Sqrt elapsed time = 1.0246 seconds.
SIQS elapsed time = 15.5372 seconds.
pretesting / qs ratio was 0.93
fac: check tune params contained invalid parameter(s), ignoring tune info.
fac: setting target pretesting digits to 27.08
    t15: 67.74
    t20: 29.98
    t25: 4.05
    t30: 0.42
    t35: 0.03
fac: estimated sum of completed work is t27.08
. . .

Edit: This is occurring on 2 of 6 machines.

chris2be8 · 2021-09-17, 15:50

Quote:

Originally Posted by EdH

On a different subject, I'm trying to figure out how to send options in the following script line:

Code:

echo "factor(${composite})" | ./yafu

I've tried adding -one and -silent before and after the pipe (at the ends of each command) and inside and outside the quotes, with and without another set of enclosing quotes. Can these be added to the script line, or can I place them in yafu.ini in some manner? I'm using separate copies of yafu and yafu.ini within the working directory where the script is located, so I'm free to modify yafu.ini to be specific. Alternately for -silent, is there a less verbose mode?

My perl scripts use:

Code:

$cmd="\"$YAFU\" 'factor($_[0])' -p -ecm_path /usr/bin/ecm -logfile $LOGFILE -threads $NUM_CPUS";
...
$res=system($cmd);

perl substitutes suitable values for $YAFU, $_[0], $LOGFILE and $NUM_CPUS, then passes the resulting string to the shell.

Or a trivial example from the command line:

Code:

chris@rigel:~/bin$ yafu 'factor(2^128+1)' -threads 2


fac: factoring 340282366920938463463374607431768211457
fac: using pretesting plan: normal
fac: using tune info for qs/gnfs crossover
div: primes less than 10000
rho: x^2 + 3, starting 1000 iterations on C39 
rho: x^2 + 2, starting 1000 iterations on C39 
rho: x^2 + 1, starting 1000 iterations on C39 
pm1: starting B1 = 150K, B2 = gmp-ecm default on C39
ecm: 30/30 curves on C39, B1=2K, B2=gmp-ecm default

starting SIQS on c39: 340282366920938463463374607431768211457

==== sieving in progress ( 2 threads):     656 relations needed ====
====            Press ctrl-c to abort and save state            ====
539 rels found: 297 full + 242 from 2231 partial, (34049.89 rels/sec)

SIQS elapsed time = 0.1017 seconds.
Total factoring time = 0.4423 seconds


***factors found***

P22 = 5704689200685129054721
P17 = 59649589127497217

ans = 1

Note it used 2 threads for sieving, so it saw the parameter -threads 2

bsquared · 2021-09-17, 18:48

Quote:

Originally Posted by EdH

Probably due to my CPUs missing (almost) all the valuable flags, I got an error after "make clean NFS=clean USE_SSE41=clean":[code]$ make NFS=1 USE_SSE41=1
. . .
factor/avx-ecm/avx_ecm.h:836:34: error: unknown type name \u2018__m512i\u2019
836 | void vecmul52_1(vec_bignum_t* a, __m512i b, vec_bignum_t* c, vec_bignum_t* n, vec_bignum_t* s, vec_monty_t* mdata);
| ^~~~~~~
. . .

Sorry if I'm a pain. Thanks for all the help.

No worries at all. You just caught me in the middle of other intermediate updates, where I don't even do my usual poor job of checking if things work on other architectures. Try the latest commit now.

Quote:

Originally Posted by EdH

Revision 2.07 appears to be stuck in a loop:

You got me with this one, no idea what's going on. Maybe some context? Is this a resume or did it run through from scratch? Running on cmd line or in interpreter? etc.

EdH · 2021-09-17, 21:01

Quote:

Originally Posted by chris2be8

My perl scripts use:

Code:

$cmd="\"$YAFU\" 'factor($_[0])' -p -ecm_path /usr/bin/ecm -logfile $LOGFILE -threads $NUM_CPUS";
...
$res=system($cmd);

perl substitutes suitable values for $YAFU, $_[0], $LOGFILE and $NUM_CPUS, then passes the resulting string to the shell.
. . .

Thanks. I'll try some more syntax variations with my script to see if I can match your perl call.

Quote:

Originally Posted by bsquared

No worries at all. You just caught me in the middle of other intermediate updates, where I don't even do my usual poor job of checking if things work on other architectures. Try the latest commit now.

You got me with this one, no idea what's going on. Maybe some context? Is this a resume or did it run through from scratch? Running on cmd line or in interpreter? etc.

Excellent! Thanks! All seems well.

Note: I found that tiny, unobtrusive, miniature, lower case, "v" you had hidden in the yafu.ini file.

So the verbosity is back to my liking.

As for the looping machines, they were running a bash script that feeds YAFU a composite. Both had already completed a good number of composites via the script. The one machine I noted had been looping for almost 7 hours overnight. I just fed that composite to the new version as a test and it flew through it. (Of course, I expected it would.)

Thanks for all!

EdH · 2021-09-18, 00:08

I seemed to still have looping on one of the same machines as before, but I did break out of it without CTRL-c:

Code:

. . .
starting SIQS on c89: 13552335418571691348826103661651040263240362626711372932187961446062171132450991836093689

==== sieving in progress ( 24 threads):   68592 relations needed ====
====             Press ctrl-c to abort and save state            ====
71951 rels found: 21038 full + 50913 from 925085 partial, (1651661.66 rels/sec)

SIQS elapsed time = 17.1809 seconds.

starting SIQS on c89: 13552335418571691348826103661651040263240362626711372932187961446062171132450991836093689

==== sieving in progress ( 24 threads):   68592 relations needed ====
====             Press ctrl-c to abort and save state            ====
71951 rels found: 21038 full + 50913 from 925085 partial, (1610592.10 rels/sec)

SIQS elapsed time = 16.3661 seconds.

starting SIQS on c89: 13552335418571691348826103661651040263240362626711372932187961446062171132450991836093689

==== sieving in progress ( 24 threads):   68592 relations needed ====
====             Press ctrl-c to abort and save state            ====
71951 rels found: 21038 full + 50913 from 925085 partial, (1650011.16 rels/sec)

SIQS elapsed time = 18.6747 seconds.
. . .

I noticed the rels found was already populated, so I used CTRL-z to pause YAFU and deleted siqs.dat. Then I resumed with "fg" and the SIQS operation started from zero rels and completed the factoring. I've added "rm siqs.dat" to my script so there shouldn't be any leftover relations from previous runs. I'll see if this machine loops anymore.

EdH · 2021-09-18, 18:39

An exciting day! I just updated all my machines, Core2 , i3, i5, i7, AMD (without SSE4_1) and Xeons. (I had to make a script.)

All are now running YAFU 2.07!!

A couple questions:

- I didn't, but should I rebuild the Core2 machines with the Core2 line in the Makefile uncommented?

- In ytools, should the "CC = gcc-7.3.0" be changed to match the "CC = gcc" in the ysieve and YAFU Makefiles?

Thanks for all!

Similar Threads
Thread	Thread Starter	Forum	Replies	Last Post
yafu ignoring yafu.ini	chris2be8	YAFU	9	2022-02-17 17:52
Running YAFU via Aliqueit doesn't find yafu.ini	EdH	YAFU	8	2018-03-14 17:22
YAFU-1.34	bsquared	YAFU	119	2015-11-05 16:24
Yafu bug.	storflyt32	YAFU	2	2015-06-29 05:19
yafu 1.32	bsquared	YAFU	28	2012-07-20 16:17

2021-09-18, 18:39	#253
EdH "Ed Hall" Dec 2009 Adirondack Mtns 10627₈ Posts	An exciting day! I just updated all my machines, Core2 , i3, i5, i7, AMD (without SSE4_1) and Xeons. (I had to make a script.) All are now running YAFU 2.07!! A couple questions: - I didn't, but should I rebuild the Core2 machines with the Core2 line in the Makefile uncommented? - In ytools, should the "CC = gcc-7.3.0" be changed to match the "CC = gcc" in the ysieve and YAFU Makefiles? Thanks for all!