mersenneforum.org  

Go Back   mersenneforum.org > Factoring Projects > YAFU

Reply
 
Thread Tools
Old 2021-09-16, 17:10   #243
bsquared
 
bsquared's Avatar
 
"Ben"
Feb 2007

DFB16 Posts
Default

Quote:
Originally Posted by EdH View Post
Just an additional note that the other Xeon acted the same. I forgot to include it in the last post.
Try the most-recent commit (just now). It is working for me...
bsquared is offline   Reply With Quote
Old 2021-09-16, 17:20   #244
bsquared
 
bsquared's Avatar
 
"Ben"
Feb 2007

3×1,193 Posts
Default

Quote:
Originally Posted by James Heinrich View Post
I'll admit I'm posting this while looking at v1.34.5 and not v2.07, but is it possible to add in some kind of rough ETA to SIQS, which currently displays something like:
Not trivial, given the ability to restart jobs and the non-linear growth of relation-sets in DLP/3LP variations.

Very roughly, when the number of fulls from partials is equal to the number of fulls then you are about 2/3 to 3/4 done.
bsquared is offline   Reply With Quote
Old 2021-09-16, 17:27   #245
James Heinrich
 
James Heinrich's Avatar
 
"James Heinrich"
May 2004
ex-Northern Ontario

2×5×353 Posts
Default

Quote:
Originally Posted by bsquared View Post
Not trivial... then you are about 2/3 to 3/4 done.
I figured it's hard to get a precise number. But just wondering if something "very approximate" could be contrived, something like "about 3-6 hours left" or something like that?
James Heinrich is offline   Reply With Quote
Old 2021-09-16, 22:18   #246
EdH
 
EdH's Avatar
 
"Ed Hall"
Dec 2009
Adirondack Mtns

3×372 Posts
Default

Quote:
Originally Posted by bsquared View Post
Try the most-recent commit (just now). It is working for me...
Probably due to my CPUs missing (almost) all the valuable flags, I got an error after "make clean NFS=clean USE_SSE41=clean":
Code:
$ make NFS=1 USE_SSE41=1
. . .
factor/avx-ecm/avx_ecm.h:836:34: error: unknown type name \u2018__m512i\u2019
  836 | void vecmul52_1(vec_bignum_t* a, __m512i b, vec_bignum_t* c, vec_bignum_t* n, vec_bignum_t* s, vec_monty_t* mdata);
      |                                  ^~~~~~~
. . .
make: *** [Makefile:393: factor/avx-ecm/avxecm.o] Error 1
I also tried without the sse4_1 option and received the same error message.


On a different subject, I'm trying to figure out how to send options in the following script line:
Code:
echo "factor(${composite})" | ./yafu
I've tried adding -one and -silent before and after the pipe (at the ends of each command) and inside and outside the quotes, with and without another set of enclosing quotes. Can these be added to the script line, or can I place them in yafu.ini in some manner? I'm using separate copies of yafu and yafu.ini within the working directory where the script is located, so I'm free to modify yafu.ini to be specific. Alternately for -silent, is there a less verbose mode?

Sorry if I'm a pain. Thanks for all the help.
EdH is offline   Reply With Quote
Old 2021-09-16, 22:33   #247
James Heinrich
 
James Heinrich's Avatar
 
"James Heinrich"
May 2004
ex-Northern Ontario

353010 Posts
Default

Quote:
Originally Posted by James Heinrich View Post
1) ECM runs as a single thread of yafu-x64.exe
Just confirming that internal ECM now uses the appropriate number of threads in v2.07 (Win10), thanks.

I did notice something when checking that -- I'm specifying -p on the commandline but yafu-x64.exe is running in "normal" priority, not idle (p is also set in yafu.ini).
Attached Thumbnails
Click image for larger version

Name:	idle.png
Views:	38
Size:	1.5 KB
ID:	25665  
James Heinrich is offline   Reply With Quote
Old 2021-09-17, 12:30   #248
EdH
 
EdH's Avatar
 
"Ed Hall"
Dec 2009
Adirondack Mtns

10000000010112 Posts
Default

Revision 2.07 appears to be stuck in a loop:
Code:
. . .
starting SIQS on c88: 1072980570200329351188730228988024470047893288038112575153917530948170108341537641314357
70449 relations found: 20728 full + 49721 from 872363 partial
threw away 0 relations with large primes too small

==== sieve params ====
n = 88 digits, 290 bits
factor base: 66560 primes (max prime = 1773997)
single large prime cutoff: 195139670 (110 * pmax)
double large prime range from 3147065356009 to 2173398172298490
DLP MFB = 1.85
using SSE41 enabled 32k sieve core
sieve interval: 12 blocks of size 32768
polynomial A has ~ 11 factors
using multiplier of 1 (kn mod 8 == 5)
using SPV correction of 20 bits, starting at offset 35
trial factoring cutoff at 93 bits

==== sieving in progress ( 24 threads):   66624 relations needed ====
====             Press ctrl-c to abort and save state            ====
70449 rels found: 20728 full + 49721 from 872363 partial, (1592093.11 rels/sec)

sieving required 0 total polynomials (4294967295 'A' polynomials)
trial division touched 0 sieve locations out of 0
dlp-ecm: 0 failures, 0 attempts, 0 outside range, 0 prp, 0 useful
total reports = 0, total surviving reports = 0
total blocks sieved = 0, avg surviving reports per block = -nan
Elapsed time: 0.5610 sec
QS elapsed time = 0.5611 seconds.

==== post processing stage (msieve-1.38) ====
read 893090 relations
begin singleton removal with 893090 relations
reduce to 162769 relations in 9 passes
attempting to read and process 162769 relations
recovered 162769 relations
recovered 129773 polynomials
attempting to build 70448 cycles
found 70448 cycles from 162769 relations in 5 passes
distribution of cycle lengths:
   length 1 : 20728
   length 2 : 15380
   length 3 : 12812
   length 4 : 8869
   length 5 : 5644
   length 6 : 3303
   length 7 : 1814
   length 9+: 1898
largest cycle: 18 relations
matrix is 66560 x 70448 (16.6 MB) with weight 3794245 (53.86/col)
sparse part has weight 3794245 (53.86/col)
filtering completed in 4 passes
matrix is 59263 x 59327 (13.3 MB) with weight 3021409 (50.93/col)
sparse part has weight 3021409 (50.93/col)
saving the first 48 matrix rows for later
matrix is 59215 x 59327 (11.6 MB) with weight 2672491 (45.05/col)
sparse part has weight 2454774 (41.38/col)
matrix includes 64 packed rows
using block size 23730 for processor cache size 15360 kB
commencing Lanczos iteration
memory use: 9.5 MB
lanczos halted after 938 iterations (dim = 59210)
recovered 14 nontrivial dependencies
Lanczos elapsed time = 14.3076 seconds.
Sqrt elapsed time = 1.3032 seconds.
SIQS elapsed time = 16.1745 seconds.
pretesting / qs ratio was 0.90
fac: check tune params contained invalid parameter(s), ignoring tune info.
fac: setting target pretesting digits to 27.08
    t15: 67.74
    t20: 29.98
    t25: 4.05
    t30: 0.42
    t35: 0.03
fac: estimated sum of completed work is t27.08

starting SIQS on c88: 1072980570200329351188730228988024470047893288038112575153917530948170108341537641314357
70449 relations found: 20728 full + 49721 from 872363 partial
threw away 0 relations with large primes too small

==== sieve params ====
n = 88 digits, 290 bits
factor base: 66560 primes (max prime = 1773997)
single large prime cutoff: 195139670 (110 * pmax)
double large prime range from 3147065356009 to 2173398172298490
DLP MFB = 1.85
using SSE41 enabled 32k sieve core
sieve interval: 12 blocks of size 32768
polynomial A has ~ 11 factors
using multiplier of 1 (kn mod 8 == 5)
using SPV correction of 20 bits, starting at offset 35
trial factoring cutoff at 93 bits

==== sieving in progress ( 24 threads):   66624 relations needed ====
====             Press ctrl-c to abort and save state            ====
70449 rels found: 20728 full + 49721 from 872363 partial, (1348970.02 rels/sec)

sieving required 0 total polynomials (4294967295 'A' polynomials)
trial division touched 0 sieve locations out of 0
dlp-ecm: 0 failures, 0 attempts, 0 outside range, 0 prp, 0 useful
total reports = 0, total surviving reports = 0
total blocks sieved = 0, avg surviving reports per block = -nan
Elapsed time: 0.6621 sec
QS elapsed time = 0.6622 seconds.

==== post processing stage (msieve-1.38) ====
read 893090 relations
begin singleton removal with 893090 relations
reduce to 162769 relations in 9 passes
attempting to read and process 162769 relations
recovered 162769 relations
recovered 129773 polynomials
attempting to build 70448 cycles
found 70448 cycles from 162769 relations in 5 passes
distribution of cycle lengths:
   length 1 : 20728
   length 2 : 15380
   length 3 : 12812
   length 4 : 8869
   length 5 : 5644
   length 6 : 3303
   length 7 : 1814
   length 9+: 1898
largest cycle: 18 relations
matrix is 66560 x 70448 (16.6 MB) with weight 3794245 (53.86/col)
sparse part has weight 3794245 (53.86/col)
filtering completed in 4 passes
matrix is 59263 x 59327 (13.3 MB) with weight 3021409 (50.93/col)
sparse part has weight 3021409 (50.93/col)
saving the first 48 matrix rows for later
matrix is 59215 x 59327 (11.6 MB) with weight 2672491 (45.05/col)
sparse part has weight 2454774 (41.38/col)
matrix includes 64 packed rows
using block size 23730 for processor cache size 15360 kB
commencing Lanczos iteration
memory use: 9.5 MB
lanczos halted after 938 iterations (dim = 59210)
recovered 14 nontrivial dependencies
Lanczos elapsed time = 13.7053 seconds.
Sqrt elapsed time = 1.0454 seconds.
SIQS elapsed time = 15.4167 seconds.
pretesting / qs ratio was 0.94
fac: check tune params contained invalid parameter(s), ignoring tune info.
fac: setting target pretesting digits to 27.08
    t15: 67.74
    t20: 29.98
    t25: 4.05
    t30: 0.42
    t35: 0.03
fac: estimated sum of completed work is t27.08

starting SIQS on c88: 1072980570200329351188730228988024470047893288038112575153917530948170108341537641314357
70449 relations found: 20728 full + 49721 from 872363 partial
threw away 0 relations with large primes too small

==== sieve params ====
n = 88 digits, 290 bits
factor base: 66560 primes (max prime = 1773997)
single large prime cutoff: 195139670 (110 * pmax)
double large prime range from 3147065356009 to 2173398172298490
DLP MFB = 1.85
using SSE41 enabled 32k sieve core
sieve interval: 12 blocks of size 32768
polynomial A has ~ 11 factors
using multiplier of 1 (kn mod 8 == 5)
using SPV correction of 20 bits, starting at offset 35
trial factoring cutoff at 93 bits

==== sieving in progress ( 24 threads):   66624 relations needed ====
====             Press ctrl-c to abort and save state            ====
70449 rels found: 20728 full + 49721 from 872363 partial, (1616530.37 rels/sec)

sieving required 0 total polynomials (4294967295 'A' polynomials)
trial division touched 0 sieve locations out of 0
dlp-ecm: 0 failures, 0 attempts, 0 outside range, 0 prp, 0 useful
total reports = 0, total surviving reports = 0
total blocks sieved = 0, avg surviving reports per block = -nan
Elapsed time: 0.5525 sec
QS elapsed time = 0.5526 seconds.

==== post processing stage (msieve-1.38) ====
read 893090 relations
begin singleton removal with 893090 relations
reduce to 162769 relations in 9 passes
attempting to read and process 162769 relations
recovered 162769 relations
recovered 129773 polynomials
attempting to build 70448 cycles
found 70448 cycles from 162769 relations in 5 passes
distribution of cycle lengths:
   length 1 : 20728
   length 2 : 15380
   length 3 : 12812
   length 4 : 8869
   length 5 : 5644
   length 6 : 3303
   length 7 : 1814
   length 9+: 1898
largest cycle: 18 relations
matrix is 66560 x 70448 (16.6 MB) with weight 3794245 (53.86/col)
sparse part has weight 3794245 (53.86/col)
filtering completed in 4 passes
matrix is 59263 x 59327 (13.3 MB) with weight 3021409 (50.93/col)
sparse part has weight 3021409 (50.93/col)
saving the first 48 matrix rows for later
matrix is 59215 x 59327 (11.6 MB) with weight 2672491 (45.05/col)
sparse part has weight 2454774 (41.38/col)
matrix includes 64 packed rows
using block size 23730 for processor cache size 15360 kB
commencing Lanczos iteration
memory use: 9.5 MB
lanczos halted after 938 iterations (dim = 59210)
recovered 14 nontrivial dependencies
Lanczos elapsed time = 13.9592 seconds.
Sqrt elapsed time = 1.0246 seconds.
SIQS elapsed time = 15.5372 seconds.
pretesting / qs ratio was 0.93
fac: check tune params contained invalid parameter(s), ignoring tune info.
fac: setting target pretesting digits to 27.08
    t15: 67.74
    t20: 29.98
    t25: 4.05
    t30: 0.42
    t35: 0.03
fac: estimated sum of completed work is t27.08
. . .
Edit: This is occurring on 2 of 6 machines.

Last fiddled with by EdH on 2021-09-17 at 12:36
EdH is offline   Reply With Quote
Old 2021-09-17, 15:50   #249
chris2be8
 
chris2be8's Avatar
 
Sep 2009

42258 Posts
Default

Quote:
Originally Posted by EdH View Post
On a different subject, I'm trying to figure out how to send options in the following script line:
Code:
echo "factor(${composite})" | ./yafu
I've tried adding -one and -silent before and after the pipe (at the ends of each command) and inside and outside the quotes, with and without another set of enclosing quotes. Can these be added to the script line, or can I place them in yafu.ini in some manner? I'm using separate copies of yafu and yafu.ini within the working directory where the script is located, so I'm free to modify yafu.ini to be specific. Alternately for -silent, is there a less verbose mode?
My perl scripts use:
Code:
$cmd="\"$YAFU\" 'factor($_[0])' -p -ecm_path /usr/bin/ecm -logfile $LOGFILE -threads $NUM_CPUS";
...
$res=system($cmd);
perl substitutes suitable values for $YAFU, $_[0], $LOGFILE and $NUM_CPUS, then passes the resulting string to the shell.

Or a trivial example from the command line:
Code:
chris@rigel:~/bin$ yafu 'factor(2^128+1)' -threads 2


fac: factoring 340282366920938463463374607431768211457
fac: using pretesting plan: normal
fac: using tune info for qs/gnfs crossover
div: primes less than 10000
rho: x^2 + 3, starting 1000 iterations on C39 
rho: x^2 + 2, starting 1000 iterations on C39 
rho: x^2 + 1, starting 1000 iterations on C39 
pm1: starting B1 = 150K, B2 = gmp-ecm default on C39
ecm: 30/30 curves on C39, B1=2K, B2=gmp-ecm default

starting SIQS on c39: 340282366920938463463374607431768211457

==== sieving in progress ( 2 threads):     656 relations needed ====
====            Press ctrl-c to abort and save state            ====
539 rels found: 297 full + 242 from 2231 partial, (34049.89 rels/sec)

SIQS elapsed time = 0.1017 seconds.
Total factoring time = 0.4423 seconds


***factors found***

P22 = 5704689200685129054721
P17 = 59649589127497217

ans = 1
Note it used 2 threads for sieving, so it saw the parameter -threads 2
chris2be8 is offline   Reply With Quote
Old 2021-09-17, 18:48   #250
bsquared
 
bsquared's Avatar
 
"Ben"
Feb 2007

3×1,193 Posts
Default

Quote:
Originally Posted by EdH View Post
Probably due to my CPUs missing (almost) all the valuable flags, I got an error after "make clean NFS=clean USE_SSE41=clean":[code]$ make NFS=1 USE_SSE41=1
. . .
factor/avx-ecm/avx_ecm.h:836:34: error: unknown type name \u2018__m512i\u2019
836 | void vecmul52_1(vec_bignum_t* a, __m512i b, vec_bignum_t* c, vec_bignum_t* n, vec_bignum_t* s, vec_monty_t* mdata);
| ^~~~~~~
. . .

Sorry if I'm a pain. Thanks for all the help.
No worries at all. You just caught me in the middle of other intermediate updates, where I don't even do my usual poor job of checking if things work on other architectures. Try the latest commit now.

Quote:
Originally Posted by EdH View Post
Revision 2.07 appears to be stuck in a loop:
You got me with this one, no idea what's going on. Maybe some context? Is this a resume or did it run through from scratch? Running on cmd line or in interpreter? etc.
bsquared is offline   Reply With Quote
Old 2021-09-17, 21:01   #251
EdH
 
EdH's Avatar
 
"Ed Hall"
Dec 2009
Adirondack Mtns

10000000010112 Posts
Default

Quote:
Originally Posted by chris2be8 View Post
My perl scripts use:
Code:
$cmd="\"$YAFU\" 'factor($_[0])' -p -ecm_path /usr/bin/ecm -logfile $LOGFILE -threads $NUM_CPUS";
...
$res=system($cmd);
perl substitutes suitable values for $YAFU, $_[0], $LOGFILE and $NUM_CPUS, then passes the resulting string to the shell.
. . .
Thanks. I'll try some more syntax variations with my script to see if I can match your perl call.

Quote:
Originally Posted by bsquared View Post
No worries at all. You just caught me in the middle of other intermediate updates, where I don't even do my usual poor job of checking if things work on other architectures. Try the latest commit now.

You got me with this one, no idea what's going on. Maybe some context? Is this a resume or did it run through from scratch? Running on cmd line or in interpreter? etc.
Excellent! Thanks! All seems well.

Note: I found that tiny, unobtrusive, miniature, lower case, "v" you had hidden in the yafu.ini file. So the verbosity is back to my liking.

As for the looping machines, they were running a bash script that feeds YAFU a composite. Both had already completed a good number of composites via the script. The one machine I noted had been looping for almost 7 hours overnight. I just fed that composite to the new version as a test and it flew through it. (Of course, I expected it would.)

Thanks for all!
EdH is offline   Reply With Quote
Old 2021-09-18, 00:08   #252
EdH
 
EdH's Avatar
 
"Ed Hall"
Dec 2009
Adirondack Mtns

3·372 Posts
Default Still Have Looping

I seemed to still have looping on one of the same machines as before, but I did break out of it without CTRL-c:
Code:
. . .
starting SIQS on c89: 13552335418571691348826103661651040263240362626711372932187961446062171132450991836093689

==== sieving in progress ( 24 threads):   68592 relations needed ====
====             Press ctrl-c to abort and save state            ====
71951 rels found: 21038 full + 50913 from 925085 partial, (1651661.66 rels/sec)

SIQS elapsed time = 17.1809 seconds.

starting SIQS on c89: 13552335418571691348826103661651040263240362626711372932187961446062171132450991836093689

==== sieving in progress ( 24 threads):   68592 relations needed ====
====             Press ctrl-c to abort and save state            ====
71951 rels found: 21038 full + 50913 from 925085 partial, (1610592.10 rels/sec)

SIQS elapsed time = 16.3661 seconds.

starting SIQS on c89: 13552335418571691348826103661651040263240362626711372932187961446062171132450991836093689

==== sieving in progress ( 24 threads):   68592 relations needed ====
====             Press ctrl-c to abort and save state            ====
71951 rels found: 21038 full + 50913 from 925085 partial, (1650011.16 rels/sec)

SIQS elapsed time = 18.6747 seconds.
. . .
I noticed the rels found was already populated, so I used CTRL-z to pause YAFU and deleted siqs.dat. Then I resumed with "fg" and the SIQS operation started from zero rels and completed the factoring. I've added "rm siqs.dat" to my script so there shouldn't be any leftover relations from previous runs. I'll see if this machine loops anymore.
EdH is offline   Reply With Quote
Old 2021-09-18, 18:39   #253
EdH
 
EdH's Avatar
 
"Ed Hall"
Dec 2009
Adirondack Mtns

410710 Posts
Default

An exciting day! I just updated all my machines, Core2 , i3, i5, i7, AMD (without SSE4_1) and Xeons. (I had to make a script.)

All are now running YAFU 2.07!!

A couple questions:

- I didn't, but should I rebuild the Core2 machines with the Core2 line in the Makefile uncommented?

- In ytools, should the "CC = gcc-7.3.0" be changed to match the "CC = gcc" in the ysieve and YAFU Makefiles?

Thanks for all!
EdH is offline   Reply With Quote
Reply

Thread Tools


Similar Threads
Thread Thread Starter Forum Replies Last Post
yafu ignoring yafu.ini chris2be8 YAFU 6 2019-10-17 16:22
Running YAFU via Aliqueit doesn't find yafu.ini EdH YAFU 8 2018-03-14 17:22
YAFU-1.34 bsquared YAFU 119 2015-11-05 16:24
Yafu bug. storflyt32 YAFU 2 2015-06-29 05:19
yafu 1.32 bsquared YAFU 28 2012-07-20 16:17

All times are UTC. The time now is 23:41.


Mon Nov 29 23:41:21 UTC 2021 up 129 days, 18:10, 0 users, load averages: 2.44, 1.61, 1.38

Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2021, Jelsoft Enterprises Ltd.

This forum has received and complied with 0 (zero) government requests for information.

Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License, Version 1.2 or any later version published by the Free Software Foundation.
A copy of the license is included in the FAQ.