mersenneforum.org

mersenneforum.org (https://www.mersenneforum.org/index.php)
-   YAFU (https://www.mersenneforum.org/forumdisplay.php?f=96)
-   -   YAFU 2.0 (https://www.mersenneforum.org/showthread.php?t=26681)

bsquared 2021-09-12 14:52

[QUOTE=James Heinrich;587752]I'm back home now and able to test v2.06 on my Win10/i8-8100 machine.
It works, but I've noticed a couple things (which entirely could be me doing something wrong).

1) ECM runs as a single thread of yafu-x64.exe
I don't care if ECM runs within yafu or spawns external ecm.exe as long as it uses the appropriate number of threads.
Relevant yafu.ini entries:
[c]threads=4[/c]
[c]ecm_path=c:\users\user\desktop\yafu2\ecm\ecm.exe[/c]
If I specify an [i]invalid[/i] path for ecm.exe then it appropriately tells me "ecm: ECM executable does not exist at c:\invalid\path\ecm.exe" and "using internal single threaded ECM...", but even when the path to ecm.exe is valid it appears to be using the internal, single-threaded ECM (with no warning).
[/QUOTE]

This is something I can fix, but in the meantime use -ext_ecm X on the command line or .ini file, where X is the B1 crossover point to start using external ecm (something like 50000 works well).

[SIZE="1"]The problem is that this value is set really big when avx-ecm is available during compilation. I need to add a check that it is also available at runtime...[/SIZE]


[QUOTE=James Heinrich;587752]
2) Number of witnesses for PRP is always "1" no even if I set something like [c]nprp=20[/c] in yafu.ini
[/QUOTE]

I will fix this. However for most factoring jobs aprcl is used instead of gmp's prp function and aprcl doesn't use or need the witnesses option.

[QUOTE=James Heinrich;587752]
3) Increased verbosity doesn't seem to work (at least in yafu.ini). The description says "Note that more v's increase the verbosity" but if I specify [c]v v[/c] it tells me "invalid option v v" and spews the list of valid options and exits[/QUOTE]

Put the v's on separate lines in the .ini file. Each line is interpreted as one option.

James Heinrich 2021-09-12 18:43

[QUOTE=bsquared;587763]The problem is that this value is set really big when avx-ecm is available during compilation. I need to add a check that it is also available at runtime...[/QUOTE]I was able to use your suggested config setting to spawn the external ecm.exe, but I'm not certain how it's supposed to work with the internal ECM -- is that not multithreaded? Is there something I'm missing to be able to use the built-in ECM but make use of available threads?

[QUOTE=bsquared;587763]Put the v's on separate lines in the .ini file. Each line is interpreted as one option.[/QUOTE]This wasn't obvious to me, perhaps an amendment to the .ini file to make that clearer would help.

kruoli 2021-09-12 18:57

On my installations, I always use [C]prefer_gmpecm[/C] and this works as intended, also with 2.06.

bsquared 2021-09-12 19:01

[QUOTE=James Heinrich;587772]I was able to use your suggested config setting to spawn the external ecm.exe, but I'm not certain how it's supposed to work with the internal ECM -- is that not multithreaded? Is there something I'm missing to be able to use the built-in ECM but make use of available threads?

This wasn't obvious to me, perhaps an amendment to the .ini file to make that clearer would help.[/QUOTE]

Before ecm version 7 only single threads were allowed, so yafu does a check on the ecm version linked in during compile. I'm pretty sure I'm linking with version 7.0.4 in the executable I provide but it's possible there is a bug in detecting that version. I'll look it over.

Yes I'll add some text to the .ini file description.

bsquared 2021-09-12 19:11

[QUOTE=kruoli;587775]On my installations, I always use [C]prefer_gmpecm[/C] and this works as intended, also with 2.06.[/QUOTE]

That should only matter if you have AVX-512, in which case yafu will default to using AVX-ECM.

bur 2021-09-13 13:06

edit: I saw this was reported earlier and should be fixed?

I pulled the newest version and did a [C]make clean[/C] and [C]make NFS=1 USE_SSE41=1 USE_AVX2=1[/C] and building quits with the following error:

[QUOTE]/usr/bin/ld: ./libyecm.a(avx_ecm_main.o): in function `vec_ecm_main':
/home/florian/Math/yafu/factor/avx-ecm/avx_ecm_main.c:592: undefined reference to `vecmulmod52_fixed1040_bfips'
/usr/bin/ld: /home/florian/Math/yafu/factor/avx-ecm/avx_ecm_main.c:593: undefined reference to `vecsqrmod52_fixed1040_bfips'
/usr/bin/ld: /home/florian/Math/yafu/factor/avx-ecm/avx_ecm_main.c:594: undefined reference to `vec_simul_addsub52_fixed1040'
[... lots of similar lines][/QUOTE]

I also tried your previous suggestion to use the [C]USE_BMI2=1[/C] flag. And I used the flags during [C]make clean[/C].

Do you know why I still run into the problem?

bsquared 2021-09-13 13:38

[QUOTE=bur;587804]

Do you know why I still run into the problem?[/QUOTE]

Because for some reason my push/commit didn't take... I'm still learning this whole git thing... :redface: I'll try again tonight.

James Heinrich 2021-09-14 01:05

I wouldn't have noticed this except for some error checking in my automated parsing of factor.log[code]09/13/21 08:12:06, ****************************
09/13/21 08:12:06, Starting factorization of 11745986472276496655662247734956521108703066873589956247227553585499089637967910706104797828678082669016193648910144450016469882935490998359642649527365479
09/13/21 08:12:06, using pretesting plan: deep
09/13/21 08:12:06, using specified qs/gnfs crossover of 100 digits
09/13/21 08:12:06, using specified qs/snfs crossover of 75 digits
09/13/21 08:12:06, ****************************
09/13/21 08:12:06, rho: x^2 + 3, starting 1000 iterations on C155
09/13/21 08:12:06, prp5 = 91291
09/13/21 08:12:06, rho: x^2 + 3, starting 1000 iterations on C150
09/13/21 08:12:06, prp7 = 7805879
09/13/21 08:12:06, rho: x^2 + 3, starting 1000 iterations on C143
09/13/21 08:12:06, rho: x^2 + 2, starting 1000 iterations on C143
09/13/21 08:12:07, rho: x^2 + 1, starting 1000 iterations on C143
09/13/21 08:12:07, pm1: starting B1 = 150K, B2 = gmp-ecm default on C143
09/13/21 08:12:07, current ECM pretesting depth: 0.00
09/13/21 08:12:07, scheduled 30 curves at B1=2000 toward target pretesting depth of 47.67
09/13/21 08:12:07, Finished 30 curves using GMP-ECM method on C143 input, B1=2k, B2=gmp-ecm default
09/13/21 08:12:07, current ECM pretesting depth: 15.18
09/13/21 08:12:07, scheduled 74 curves at B1=11000 toward target pretesting depth of 47.67
09/13/21 08:12:10, Finished 74 curves using GMP-ECM method on C143 input, B1=11k, B2=gmp-ecm default
09/13/21 08:12:10, current ECM pretesting depth: 20.24
09/13/21 08:12:10, scheduled 214 curves at B1=50000 toward target pretesting depth of 47.67
09/13/21 08:12:11, prp26 = 23304102950704118347000297 (curve 2 stg2 B1=50000 sigma=3874399757 thread=1)
09/13/21 08:12:11, Finished 4 curves using GMP-ECM method on C143 input, B1=50k, B2=gmp-ecm default
09/13/21 08:12:11, current ECM pretesting depth: 20.34
09/13/21 08:12:11, scheduled 210 curves at B1=50000 toward target pretesting depth of 39.00
09/13/21 08:12:22, Finished 210 curves using GMP-ECM method on C117 input, B1=50k, B2=gmp-ecm default
09/13/21 08:12:22, pm1: starting B1 = 3750K, B2 = gmp-ecm default on C117
09/13/21 08:12:23, current ECM pretesting depth: 25.33
09/13/21 08:12:23, scheduled 430 curves at B1=250000 toward target pretesting depth of 39.00
09/13/21 08:13:47, Finished 430 curves using GMP-ECM method on C117 input, B1=250k, B2=gmp-ecm default
09/13/21 08:13:47, pm1: starting B1 = 15M, B2 = gmp-ecm default on C117
09/13/21 08:13:50, current ECM pretesting depth: 30.45
09/13/21 08:13:50, scheduled 904 curves at B1=1000000 toward target pretesting depth of 39.00
09/13/21 08:25:10, Finished 904 curves using GMP-ECM method on C117 input, B1=1M, B2=gmp-ecm default
09/13/21 08:25:10, current ECM pretesting depth: 35.56
09/13/21 08:25:10, scheduled 1619 curves at B1=3000000 toward target pretesting depth of 39.00
09/13/21 08:31:42, prp35 = 10458847902056956740364765053965417 (curve 195 stg2 B1=3000000 sigma=2620919999 thread=0)
09/13/21 08:31:49, Finished 197 curves using GMP-ECM method on C117 input, B1=3M, B2=gmp-ecm default
09/13/21 08:31:49, final ECM pretested depth: 35.98
09/13/21 08:31:49, prp84 cofactor = 67627522805179106773859180025103681923504306209525858835440580638970766947677849939
09/13/21 08:31:49, Total factoring time = 1183.1567 seconds[/code]Note the second-last line:
[c]prp84 cofactor = 67627522805179106773859180025103681923504306209525858835440580638970766947677849939[/c]
That number is only 8[b]3[/b] characters long, but claims to be a prp8[b]4[/b].

For comparison, this is what it shows on v1.34.5:[code]09/13/21 20:25:23 v1.34.5 @ 3930K, ****************************
09/13/21 20:25:23 v1.34.5 @ 3930K, Starting factorization of 11745986472276496655662247734956521108703066873589956247227553585499089637967910706104797828678082669016193648910144450016469882935490998359642649527365479
09/13/21 20:25:23 v1.34.5 @ 3930K, using pretesting plan: deep
09/13/21 20:25:23 v1.34.5 @ 3930K, using tune info for qs/gnfs crossover
09/13/21 20:25:23 v1.34.5 @ 3930K, ****************************
09/13/21 20:25:23 v1.34.5 @ 3930K, rho: x^2 + 3, starting 1000 iterations on C155
09/13/21 20:25:23 v1.34.5 @ 3930K, rho: x^2 + 2, starting 1000 iterations on C155
09/13/21 20:25:23 v1.34.5 @ 3930K, prp5 = 91291
09/13/21 20:25:23 v1.34.5 @ 3930K, rho: x^2 + 2, starting 1000 iterations on C150
09/13/21 20:25:23 v1.34.5 @ 3930K, prp7 = 7805879
09/13/21 20:25:23 v1.34.5 @ 3930K, rho: x^2 + 2, starting 1000 iterations on C143
09/13/21 20:25:23 v1.34.5 @ 3930K, rho: x^2 + 1, starting 1000 iterations on C143
09/13/21 20:25:23 v1.34.5 @ 3930K, pm1: starting B1 = 150K, B2 = gmp-ecm default on C143
09/13/21 20:25:23 v1.34.5 @ 3930K, current ECM pretesting depth: 0.00
09/13/21 20:25:23 v1.34.5 @ 3930K, scheduled 30 curves at B1=2000 toward target pretesting depth of 47.67
09/13/21 20:25:23 v1.34.5 @ 3930K, Finished 30 curves using Lenstra ECM method on C143 input, B1=2K, B2=gmp-ecm default
09/13/21 20:25:23 v1.34.5 @ 3930K, current ECM pretesting depth: 15.18
09/13/21 20:25:23 v1.34.5 @ 3930K, scheduled 74 curves at B1=11000 toward target pretesting depth of 47.67
09/13/21 20:25:27 v1.34.5 @ 3930K, Finished 74 curves using Lenstra ECM method on C143 input, B1=11K, B2=gmp-ecm default
09/13/21 20:25:27 v1.34.5 @ 3930K, current ECM pretesting depth: 20.24
09/13/21 20:25:27 v1.34.5 @ 3930K, scheduled 214 curves at B1=50000 toward target pretesting depth of 47.67
09/13/21 20:25:35 v1.34.5 @ 3930K, prp26 = 23304102950704118347000297 (curve 22 stg2 B1=50000 sigma=3377483298 thread=2)
09/13/21 20:25:35 v1.34.5 @ 3930K, Finished 132 curves using Lenstra ECM method on C143 input, B1=50K, B2=gmp-ecm default
09/13/21 20:25:35 v1.34.5 @ 3930K, current ECM pretesting depth: 23.33
09/13/21 20:25:35 v1.34.5 @ 3930K, scheduled 82 curves at B1=50000 toward target pretesting depth of 39.00
09/13/21 20:25:39 v1.34.5 @ 3930K, Finished 84 curves using Lenstra ECM method on C117 input, B1=50K, B2=gmp-ecm default
09/13/21 20:25:39 v1.34.5 @ 3930K, pm1: starting B1 = 3750K, B2 = gmp-ecm default on C117
09/13/21 20:25:41 v1.34.5 @ 3930K, current ECM pretesting depth: 25.34
09/13/21 20:25:41 v1.34.5 @ 3930K, scheduled 430 curves at B1=250000 toward target pretesting depth of 39.00
09/13/21 20:27:04 v1.34.5 @ 3930K, Finished 432 curves using Lenstra ECM method on C117 input, B1=250K, B2=gmp-ecm default
09/13/21 20:27:04 v1.34.5 @ 3930K, pm1: starting B1 = 15M, B2 = gmp-ecm default on C117
09/13/21 20:27:10 v1.34.5 @ 3930K, current ECM pretesting depth: 30.46
09/13/21 20:27:10 v1.34.5 @ 3930K, scheduled 904 curves at B1=1000000 toward target pretesting depth of 39.00
09/13/21 20:30:19 v1.34.5 @ 3930K, prp35 = 10458847902056956740364765053965417 (curve 40 stg2 B1=1000000 sigma=886406094 thread=1)
09/13/21 20:30:19 v1.34.5 @ 3930K, Finished 240 curves using Lenstra ECM method on C117 input, B1=1M, B2=gmp-ecm default
09/13/21 20:30:19 v1.34.5 @ 3930K, final ECM pretested depth: 31.78
09/13/21 20:30:19 v1.34.5 @ 3930K, scheduler: switching to sieve method
09/13/21 20:30:19 v1.34.5 @ 3930K, prp83 = 67627522805179106773859180025103681923504306209525858835440580638970766947677849939
09/13/21 20:30:19 v1.34.5 @ 3930K, Total factoring time = 296.4042 seconds[/code]

bur 2021-09-14 12:31

[QUOTE]Because for some reason my push/commit didn't take...[/QUOTE]Thanks!

bsquared 2021-09-15 01:19

2.07 now available
 
[QUOTE=James Heinrich;587832]I wouldn't have noticed this except for some error checking in my automated parsing of factor.log

Note the second-last line:
[c]prp84 cofactor = 67627522805179106773859180025103681923504306209525858835440580638970766947677849939[/c]
That number is only 8[b]3[/b] characters long, but claims to be a prp8[b]4[/b].
[/QUOTE]

This should be fixed now, as well as the previous problem building on avx2.

I verified the latest commits made it to github, we are now at 2.07.

In this version I've committed changes to the MSVC project files and linux makefile that revert back to the "normal" directory structure of yafu and its dependencies (no more ".git" or "trunk" in the relative paths of libraries). And removed some confusing stuff in the makefile. Hopefully working toward something that is easier to build.

Two other things of note:
1) I was made aware that aprcl wasn't running on factors found, this is now fixed and factors displayed to screen should now be "P" instead of "PRP"
2) AVX-ECM below 1040 bits is about 10% faster.

Regarding the "P" vs. "PRP" labeling... the .log file still generally gets "prp" printed. Which could be fixed, but before I do that I wanted to see if that would break anyone's scripting/parsing?

James Heinrich 2021-09-15 10:41

[QUOTE=bsquared;587887]Regarding the "P" vs. "PRP" labeling... the .log file still generally gets "prp" printed. Which could be fixed, but before I do that I wanted to see if that would break anyone's scripting/parsing?[/QUOTE]It wouldn't break anything I do, my regex already looks like [c]^(prp|p)([0-9]+)( cofactor)? = ([0-9]+)[/c]

We already have to handle the different format for small factors: [c]div: found prime factor = 3[/c]
Possibly this could be changed to standardize it such that all factors follow the format [c](p|prp|c)<len> = <factor>[/c]


All times are UTC. The time now is 07:37.

Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2022, Jelsoft Enterprises Ltd.