mersenneforum.org  

Go Back   mersenneforum.org > Factoring Projects > YAFU

Reply
 
Thread Tools
Old 2020-10-30, 21:08   #45
mathwiz
 
Mar 2019

24×32 Posts
Default

Quote:
Originally Posted by bsquared View Post
I've checked the latest code into r393 if anyone wants to do some testing. You will need a avx512 capable cpu.
Is https://github.com/bbuhrow/avx-ecm/ the latest repo? Unless I'm reading wrong, the only commit recently (3 days ago) was just to README.md?
mathwiz is offline   Reply With Quote
Old 2020-10-30, 21:13   #46
bsquared
 
bsquared's Avatar
 
"Ben"
Feb 2007

3,371 Posts
Default

Quote:
Originally Posted by mathwiz View Post
Is https://github.com/bbuhrow/avx-ecm/ the latest repo? Unless I'm reading wrong, the only commit recently (3 days ago) was just to README.md?
Ah, good question. Sorry, I've folded it all into yafu, so it is yafu r393 that you'll need. It is probably a good idea to update the avx-ecm repo as well since that is significantly easier to build. I can do that tonight, hopefully.
bsquared is offline   Reply With Quote
Old 2020-10-30, 22:14   #47
VBCurtis
 
VBCurtis's Avatar
 
"Curtis"
Feb 2005
Riverside, CA

52×11×17 Posts
Default

One more reason to buy one of those Phi barebones systems!

Nice work.
VBCurtis is offline   Reply With Quote
Old 2020-10-30, 22:50   #48
mathwiz
 
Mar 2019

24·32 Posts
Default

Quote:
Originally Posted by bsquared View Post
Ah, good question. Sorry, I've folded it all into yafu, so it is yafu r393 that you'll need. It is probably a good idea to update the avx-ecm repo as well since that is significantly easier to build. I can do that tonight, hopefully.
Gotcha. Is there an easy way to build just AVX-ECM from the latest yafu sources? Or, if you could update the avx-ecm git repo, that'd be great too!

I tried:

Code:
$ svn checkout https://svn.code.sf.net/p/yafu/code/branches/wip yafu-code
$ make -j 8 SKYLAKEX=1
But I get some linker errors like:

Code:
gcc-9: error: /sppdg/scratch/buhrow/projects/gmp_install/gmp-6.2.0/lib/libgmp.a: No such file or directory
mathwiz is offline   Reply With Quote
Old 2020-10-31, 00:36   #49
bsquared
 
bsquared's Avatar
 
"Ben"
Feb 2007

1101001010112 Posts
Default

Quote:
Originally Posted by VBCurtis View Post
One more reason to buy one of those Phi barebones systems!

Nice work.
Thanks!

I actually have some data for the KNL, linearly scaled from a run with B1=7e6.

7210 KNL:
2048 simultaneous curves with 256 threads
354 hours/run
10.4 minutes/curve

7250 KNL
2176 simultaneous curves with 272 threads
353 hours/run
9.7 minutes/curve

B2 then takes about 3/4 of that to go up to 100*B1, another 264 hours, so that's a lengthy ~25 day runtime per batch. But you get 2000+ curves at the end of it (B1=7G).
[edit]
Running with fewer threads and curves trades off throughput for lower wall-clock time, if that is a concern. I don't have data for the exact trades there, yet. Also, it will save intermediate checkpoint files every 100M primes for each curve. The savefiles can be fed to GMP-ECM for stage 2, if you'd rather run stage 2 that way. Run with -saveB1 on the command line to enable saving.

Memory use is pretty minimal for both stages; won't be a problem to stay in MCDRAM.

Last fiddled with by bsquared on 2020-10-31 at 00:56
bsquared is offline   Reply With Quote
Old 2020-10-31, 00:50   #50
bsquared
 
bsquared's Avatar
 
"Ben"
Feb 2007

64538 Posts
Default

Quote:
Originally Posted by mathwiz View Post
Gotcha. Is there an easy way to build just AVX-ECM from the latest yafu sources? Or, if you could update the avx-ecm git repo, that'd be great too!

I tried:

Code:
$ svn checkout https://svn.code.sf.net/p/yafu/code/branches/wip yafu-code
$ make -j 8 SKYLAKEX=1
But I get some linker errors like:
No easy way to just build part of yafu. Makefiles are not my strong suit. I very much recommend EdH's guides on building yafu. It requires a small amount of makefile editing to point to your msieve, ecm, and gmp locations.

IMO there are reasons to prefer the yafu version of avx-ecm. It integrates into factor(), so it's fire and forget on any input. It will auto-recognize Mersennes and determine the fastest approach (sometimes it's better to just use Montgomery arithmetic). It uses the NFS algebraic factor extraction framework that's already there, so you could feed it, say, (2^1265-1) / 121441 / 533831 / 136149590957161 / 556590676780973231 and it would pull out all the annoying algebraic factors and go to work on the remaining C223. You can tell it to pretest up to a specific t-level, etc.

avx-ecm has none of that, but I will update it with the new vector math at least, with maybe a command line switch to turn it on.
bsquared is offline   Reply With Quote
Old 2020-11-01, 04:38   #51
wombatman
I moo ablest echo power!
 
wombatman's Avatar
 
May 2013

1,741 Posts
Default

Posting here only because the issue seems to be related to avx-ecm. I'm trying to build the latest revision of YAFU, and no matter what I pass during make, it's failing with errors like:

Code:
/usr/bin/ld: factor/avx-ecm/avx_ecm_main.o: in function `vec_ecm_main':
/home/wombat/yafu-code/branches/wip/factor/avx-ecm/avx_ecm_main.c:266: undefined reference to `snfs_init'
/usr/bin/ld: /home/wombat/yafu-code/branches/wip/factor/avx-ecm/avx_ecm_main.c:272: undefined reference to `find_primitive_factor'
/usr/bin/ld: /home/wombat/yafu-code/branches/wip/factor/avx-ecm/avx_ecm_main.c:279: undefined reference to `snfs_clear'
/usr/bin/ld: /home/wombat/yafu-code/branches/wip/factor/avx-ecm/avx_ecm_main.c:464: undefined reference to `vecmulmod52_mersenne'
/usr/bin/ld: /home/wombat/yafu-code/branches/wip/factor/avx-ecm/avx_ecm_main.c:465: undefined reference to `vecsqrmod52_mersenne'
collect2: error: ld returned 1 exit status
make: *** [Makefile:397: all] Error 1
I'm passing in this:

Code:
 make clean && make ECM=1 USE_SSE41=1
This error is recent, though I can't say exactly which revision I was able to successfully build. Any suggestions?
wombatman is offline   Reply With Quote
Old 2020-11-01, 15:10   #52
bsquared
 
bsquared's Avatar
 
"Ben"
Feb 2007

3,371 Posts
Default

Quote:
Originally Posted by wombatman View Post
Posting here only because the issue seems to be related to avx-ecm. I'm trying to build the latest revision of YAFU, and no matter what I pass during make, it's failing with errors like:

Code:
/usr/bin/ld: factor/avx-ecm/avx_ecm_main.o: in function `vec_ecm_main':
/home/wombat/yafu-code/branches/wip/factor/avx-ecm/avx_ecm_main.c:266: undefined reference to `snfs_init'
/usr/bin/ld: /home/wombat/yafu-code/branches/wip/factor/avx-ecm/avx_ecm_main.c:272: undefined reference to `find_primitive_factor'
/usr/bin/ld: /home/wombat/yafu-code/branches/wip/factor/avx-ecm/avx_ecm_main.c:279: undefined reference to `snfs_clear'
/usr/bin/ld: /home/wombat/yafu-code/branches/wip/factor/avx-ecm/avx_ecm_main.c:464: undefined reference to `vecmulmod52_mersenne'
/usr/bin/ld: /home/wombat/yafu-code/branches/wip/factor/avx-ecm/avx_ecm_main.c:465: undefined reference to `vecsqrmod52_mersenne'
collect2: error: ld returned 1 exit status
make: *** [Makefile:397: all] Error 1
I'm passing in this:

Code:
 make clean && make ECM=1 USE_SSE41=1
This error is recent, though I can't say exactly which revision I was able to successfully build. Any suggestions?
Looks like you just need to use more makefile options. You need at least NFS=1 to bring in the nfs stuff and SKYLAKEX=1 for the avx512 vector arithmetic stuff. And if you use SKYLAKEX=1 then you also need USE_AVX2=1. My typical build line looks like this:

make NFS=1 USE_AVX2=1 USE_BMI2=1 SKYLAKEX=1

also, ECM=1 is a msieve thing, not a yafu thing, so you can leave that off.
bsquared is offline   Reply With Quote
Old 2020-11-01, 15:30   #53
bsquared
 
bsquared's Avatar
 
"Ben"
Feb 2007

3,371 Posts
Default

Another quick usage note...
Normally, yafu will switch over from using an internal GMP-ECM to an external GMP-ECM (pointed to by the ecm_path variable in yafu.ini) at B1=50k. With AVX-ECM enabled, by default that switchover happens at B1=40M instead. If you want to use AVX-ECM at higher B1, put that crossover value into the variable "ext_ecm" in yafu.ini.

For example if I want to use B1=110M with AVX-ECM and not with the external GMP-ECM, set ext_ecm=120000000, or anything larger than 110M.
bsquared is offline   Reply With Quote
Old 2020-11-01, 15:30   #54
mathwiz
 
Mar 2019

2208 Posts
Default

(waits patiently for the old avx-ecm git repo to be updated with the latest code... it's so much easier to build :)
mathwiz is offline   Reply With Quote
Old 2020-11-01, 16:56   #55
bsquared
 
bsquared's Avatar
 
"Ben"
Feb 2007

1101001010112 Posts
Default

Quote:
Originally Posted by mathwiz View Post
(waits patiently for the old avx-ecm git repo to be updated with the latest code... it's so much easier to build :)
Thanks for your patience

Now updated, tested, seems to work. Also checked in a pre-built version for windows. The mingw compiler doesn't do as good of a job, but something for windows is better than nothing.
bsquared is offline   Reply With Quote
Reply

Thread Tools


All times are UTC. The time now is 08:19.

Tue Mar 2 08:19:59 UTC 2021 up 89 days, 4:31, 0 users, load averages: 1.28, 1.35, 1.39

Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2021, Jelsoft Enterprises Ltd.

This forum has received and complied with 0 (zero) government requests for information.

Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License, Version 1.2 or any later version published by the Free Software Foundation.
A copy of the license is included in the FAQ.